Max level shown:
12024-11-28T02:00:03.318ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 9045d70a-bd91-4724-82e2-3d3e6fb40d2d roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
22024-11-28T02:00:03.318ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 9045d70a-bd91-4724-82e2-3d3e6fb40d2d resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
32024-11-28T02:00:03.321ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2923 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 9045d70a-bd91-4724-82e2-3d3e6fb40d2d response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
42024-11-28T02:00:04.194ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1587 local_addr = [fd00:1122:3344:101::b]:55621 method = GET remote_addr = [fd00:1122:3344:101::d]:56417 req_id = 3bbdf14a-54e2-43fd-99cd-ac102b749ed3 response_code = 200 uri = /2469a748-bcd2-4e0d-99cc-f79def0452cc
52024-11-28T02:00:04.345ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = f203e658-df29-4c8b-9b79-cd24e5457a5c roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
62024-11-28T02:00:04.346ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = f203e658-df29-4c8b-9b79-cd24e5457a5c resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
72024-11-28T02:00:04.347ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2054 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = f203e658-df29-4c8b-9b79-cd24e5457a5c response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
82024-11-28T02:00:05.366ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 60cfe634-9d1e-4f4f-b21c-1b1256821dd3 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
92024-11-28T02:00:05.367ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 60cfe634-9d1e-4f4f-b21c-1b1256821dd3 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
102024-11-28T02:00:05.368ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1611 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 60cfe634-9d1e-4f4f-b21c-1b1256821dd3 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
112024-11-28T02:00:06.386ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 8d62ca10-3ae6-4ed3-935c-5cdd3532370c roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
122024-11-28T02:00:06.387ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 8d62ca10-3ae6-4ed3-935c-5cdd3532370c resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
132024-11-28T02:00:06.388ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2063 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 8d62ca10-3ae6-4ed3-935c-5cdd3532370c response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
142024-11-28T02:00:07.407ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 15125c78-6cf3-4af5-bc7e-ea66ea653e83 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
152024-11-28T02:00:07.408ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 15125c78-6cf3-4af5-bc7e-ea66ea653e83 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
162024-11-28T02:00:07.410ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2310 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 15125c78-6cf3-4af5-bc7e-ea66ea653e83 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
172024-11-28T02:00:08.429ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 947197ff-0d38-49b9-ae95-22a4cf6a6a3c roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
182024-11-28T02:00:08.429ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 947197ff-0d38-49b9-ae95-22a4cf6a6a3c resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
192024-11-28T02:00:08.431ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1713 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 947197ff-0d38-49b9-ae95-22a4cf6a6a3c response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
202024-11-28T02:00:09.185ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:55495 req_id = 96835e39-97e1-4653-953d-02307166f2f9 roles = RoleSet { roles: {} } uri = /metrics/collectors/119eed64-2e24-4d5f-8c42-be6cd8bec8b5/producers?sort_by=id_ascending
212024-11-28T02:00:09.185ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:55495 req_id = 96835e39-97e1-4653-953d-02307166f2f9 resource = Database result = Ok(()) uri = /metrics/collectors/119eed64-2e24-4d5f-8c42-be6cd8bec8b5/producers?sort_by=id_ascending
222024-11-28T02:00:09.187ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2638 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:55495 req_id = 96835e39-97e1-4653-953d-02307166f2f9 response_code = 200 uri = /metrics/collectors/119eed64-2e24-4d5f-8c42-be6cd8bec8b5/producers?sort_by=id_ascending
232024-11-28T02:00:09.188ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:55495 req_id = 8af3d5ff-0ed9-449b-aadd-2d03df0a3e2b roles = RoleSet { roles: {} } uri = /metrics/collectors/119eed64-2e24-4d5f-8c42-be6cd8bec8b5/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJjNDM0YjQ0OC01MTk5LTQ2NmUtYWYzMi00ZjYyOWUzMWM3YjEifX0%3D
242024-11-28T02:00:09.188ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:55495 req_id = 8af3d5ff-0ed9-449b-aadd-2d03df0a3e2b resource = Database result = Ok(()) uri = /metrics/collectors/119eed64-2e24-4d5f-8c42-be6cd8bec8b5/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJjNDM0YjQ0OC01MTk5LTQ2NmUtYWYzMi00ZjYyOWUzMWM3YjEifX0%3D
252024-11-28T02:00:09.190ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2530 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:55495 req_id = 8af3d5ff-0ed9-449b-aadd-2d03df0a3e2b response_code = 200 uri = /metrics/collectors/119eed64-2e24-4d5f-8c42-be6cd8bec8b5/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJjNDM0YjQ0OC01MTk5LTQ2NmUtYWYzMi00ZjYyOWUzMWM3YjEifX0%3D
262024-11-28T02:00:09.449ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = da4f34c8-1b34-4904-aea1-d72f5fb20a7e roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
272024-11-28T02:00:09.449ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = da4f34c8-1b34-4904-aea1-d72f5fb20a7e resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
282024-11-28T02:00:09.450ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1954 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = da4f34c8-1b34-4904-aea1-d72f5fb20a7e response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
292024-11-28T02:00:10.471ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = e287341f-1c04-46f4-9404-f72264b788a4 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
302024-11-28T02:00:10.471ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = e287341f-1c04-46f4-9404-f72264b788a4 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
312024-11-28T02:00:10.473ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2417 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = e287341f-1c04-46f4-9404-f72264b788a4 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
322024-11-28T02:00:10.477ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 9 reason = Timeout
332024-11-28T02:00:10.477ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 15 reason = Timeout
342024-11-28T02:00:10.478ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
352024-11-28T02:00:10.478ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
362024-11-28T02:00:10.478ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 9 reason = Timeout
372024-11-28T02:00:10.478ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
382024-11-28T02:00:10.478ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
392024-11-28T02:00:10.478ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 15 reason = Timeout
402024-11-28T02:00:10.478ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 15 reason = Timeout
412024-11-28T02:00:10.478ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 15 reason = Timeout
422024-11-28T02:00:10.478ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
432024-11-28T02:00:10.478ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
442024-11-28T02:00:10.478ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
452024-11-28T02:00:10.478ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 8 reason = Timeout
462024-11-28T02:00:10.478ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2024-11-28 01:50:10.478035914 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
472024-11-28T02:00:10.478ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
482024-11-28T02:00:10.478ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 10 reason = Timeout
492024-11-28T02:00:10.478ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 15 reason = Timeout
502024-11-28T02:00:10.478ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 9 reason = Timeout
512024-11-28T02:00:10.478ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
522024-11-28T02:00:10.478ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 8 reason = Timeout
532024-11-28T02:00:10.478ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
542024-11-28T02:00:10.478ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 10 reason = Timeout
552024-11-28T02:00:10.478ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 16 reason = Timeout
562024-11-28T02:00:10.478ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
572024-11-28T02:00:10.478ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 15 reason = Timeout
582024-11-28T02:00:10.478ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
592024-11-28T02:00:10.478ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 15 reason = Timeout
602024-11-28T02:00:10.478ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
612024-11-28T02:00:10.478ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 9 reason = Timeout
622024-11-28T02:00:10.478ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:83 reason = blueprint disabled target_id = 67ac247f-c934-4c78-857a-fba4fbeb9937
632024-11-28T02:00:10.478ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 15.23\u{b5}s iteration = 9 status = {"enabled":false,"target_id":"67ac247f-c934-4c78-857a-fba4fbeb9937"}
642024-11-28T02:00:10.478ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 8 reason = Timeout
652024-11-28T02:00:10.478ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
662024-11-28T02:00:10.478ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 9 reason = Timeout
672024-11-28T02:00:10.478ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 15 reason = Timeout
682024-11-28T02:00:10.478ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
692024-11-28T02:00:10.478ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 43 reason = Timeout
702024-11-28T02:00:10.478ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 18 reason = Timeout
712024-11-28T02:00:10.478ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
722024-11-28T02:00:10.478ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
732024-11-28T02:00:10.478ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
742024-11-28T02:00:10.478ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 8 reason = Timeout
752024-11-28T02:00:10.478ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:56
762024-11-28T02:00:10.478ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
772024-11-28T02:00:10.478ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 15 reason = Timeout
782024-11-28T02:00:10.478ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
792024-11-28T02:00:10.478ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
802024-11-28T02:00:10.478ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 8 reason = Timeout
812024-11-28T02:00:10.478ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
822024-11-28T02:00:10.478ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 15 reason = Timeout
832024-11-28T02:00:10.478ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 15 reason = Timeout
842024-11-28T02:00:10.478ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
852024-11-28T02:00:10.478ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
862024-11-28T02:00:10.478ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
872024-11-28T02:00:10.478ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 9 reason = Timeout
882024-11-28T02:00:10.478ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
892024-11-28T02:00:10.478ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 43 reason = Timeout
902024-11-28T02:00:10.478ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
912024-11-28T02:00:10.478ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 15 reason = Timeout
922024-11-28T02:00:10.478ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
932024-11-28T02:00:10.478ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
942024-11-28T02:00:10.478ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 15 reason = Timeout
952024-11-28T02:00:10.478ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
962024-11-28T02:00:10.478ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
972024-11-28T02:00:10.478ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
982024-11-28T02:00:10.478ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
992024-11-28T02:00:10.478ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
1002024-11-28T02:00:10.478ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
1012024-11-28T02:00:10.478ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
1022024-11-28T02:00:10.478ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
1032024-11-28T02:00:10.478ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
1042024-11-28T02:00:10.478ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
1052024-11-28T02:00:10.478ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
1062024-11-28T02:00:10.478ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
1072024-11-28T02:00:10.478ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
1082024-11-28T02:00:10.478ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
1092024-11-28T02:00:10.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
1102024-11-28T02:00:10.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
1112024-11-28T02:00:10.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
1122024-11-28T02:00:10.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
1132024-11-28T02:00:10.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
1142024-11-28T02:00:10.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
1152024-11-28T02:00:10.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
1162024-11-28T02:00:10.479ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("36d4f615-7461-4364-960c-89ece3aaabaf.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("eb274350-c0b0-4bce-9c8a-502880f2c905.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("eb274350-c0b0-4bce-9c8a-502880f2c905.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::8))) }], valid_until: Instant { tv_sec: 1276, tv_nsec: 950325361 } })
1172024-11-28T02:00:10.479ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("41ea4a99-3038-474e-bf28-2ad338f3243f.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("9402a51b-d30f-4544-93ec-0f5dc7a485e4.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("d23bc5f2-6b92-4992-bdd2-8695e5bf9c0f.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("d23bc5f2-6b92-4992-bdd2-8695e5bf9c0f.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:1::1))) }], valid_until: Instant { tv_sec: 1276, tv_nsec: 950321441 } })
1182024-11-28T02:00:10.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
1192024-11-28T02:00:10.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
1202024-11-28T02:00:10.479ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } dns_group = external
1212024-11-28T02:00:10.479ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.343263ms iteration = 9 status = {"addresses":["[fd00:1122:3344:101::9]:5353","[fd00:1122:3344:101::8]:5353"]}
1222024-11-28T02:00:10.479ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } dns_group = internal
1232024-11-28T02:00:10.479ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.622828ms iteration = 9 status = {"addresses":["[fd00:1122:3344:3::1]:5353","[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:1::1]:5353"]}
1242024-11-28T02:00:10.479ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
1252024-11-28T02:00:10.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
1262024-11-28T02:00:10.481ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
1272024-11-28T02:00:10.481ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
1282024-11-28T02:00:10.483ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 5.380413ms iteration = 15 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
1292024-11-28T02:00:10.484ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2024-11-28 01:50:10.478035914 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
1302024-11-28T02:00:10.484ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 6.357581ms iteration = 8 status = {"expiration":"2024-11-28T01:50:10.478035914Z","pruned":[]}
1312024-11-28T02:00:10.485ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 7.114844ms iteration = 8 status = {"errors":[],"found_port_ok":[]}
1322024-11-28T02:00:10.485ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1332024-11-28T02:00:10.489ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
1342024-11-28T02:00:10.489ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
1352024-11-28T02:00:10.489ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
1362024-11-28T02:00:10.496ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1372024-11-28T02:00:10.498ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-313847b3-d670-4167-9b09-9e7d3dfcb2a4.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-313847b3-d670-4167-9b09-9e7d3dfcb2a4.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1276, tv_nsec: 969633787 } })
1382024-11-28T02:00:10.498ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1103
1392024-11-28T02:00:10.499ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
1402024-11-28T02:00:10.499ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
1412024-11-28T02:00:10.499ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
1422024-11-28T02:00:10.501ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-313847b3-d670-4167-9b09-9e7d3dfcb2a4.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-313847b3-d670-4167-9b09-9e7d3dfcb2a4.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1276, tv_nsec: 972587978 } })
1432024-11-28T02:00:10.501ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1442024-11-28T02:00:10.501ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1452024-11-28T02:00:10.501ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1103
1462024-11-28T02:00:10.505ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
1472024-11-28T02:00:10.505ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
1482024-11-28T02:00:10.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
1492024-11-28T02:00:10.505ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
1502024-11-28T02:00:10.506ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
1512024-11-28T02:00:10.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
1522024-11-28T02:00:10.508ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
1532024-11-28T02:00:10.508ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1542024-11-28T02:00:10.508ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
1552024-11-28T02:00:10.508ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1562024-11-28T02:00:10.510ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
1572024-11-28T02:00:10.510ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
1582024-11-28T02:00:10.512ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
1592024-11-28T02:00:10.513ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
1602024-11-28T02:00:10.516ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
1612024-11-28T02:00:10.516ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
1622024-11-28T02:00:10.544ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 66.082468ms iteration = 15 status = {"errors":[],"records_set_to_done":[]}
1632024-11-28T02:00:10.546ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1642024-11-28T02:00:10.547ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
1652024-11-28T02:00:10.548ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
1662024-11-28T02:00:10.549ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
1672024-11-28T02:00:10.549ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
1682024-11-28T02:00:10.549ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
1692024-11-28T02:00:10.552ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1702024-11-28T02:00:10.553ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
1712024-11-28T02:00:10.554ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1722024-11-28T02:00:10.554ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
1732024-11-28T02:00:10.555ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1742024-11-28T02:00:10.556ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 67ac247f-c934-4c78-857a-fba4fbeb9937, lookup_type: ById(67ac247f-c934-4c78-857a-fba4fbeb9937) } result = Ok(())
1752024-11-28T02:00:10.556ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
1762024-11-28T02:00:10.557ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
1772024-11-28T02:00:10.557ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
1782024-11-28T02:00:10.557ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
1792024-11-28T02:00:10.557ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
1802024-11-28T02:00:10.557ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
1812024-11-28T02:00:10.557ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
1822024-11-28T02:00:10.558ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
1832024-11-28T02:00:10.564ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 86.600055ms iteration = 15 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
1842024-11-28T02:00:10.566ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1852024-11-28T02:00:10.568ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
1862024-11-28T02:00:10.569ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
1872024-11-28T02:00:10.570ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
1882024-11-28T02:00:10.570ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
1892024-11-28T02:00:10.570ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
1902024-11-28T02:00:10.571ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
1912024-11-28T02:00:10.571ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
1922024-11-28T02:00:10.571ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 93.25972ms iteration = 43 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
1932024-11-28T02:00:10.573ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
1942024-11-28T02:00:10.573ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:171 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
1952024-11-28T02:00:10.573ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 95.399877ms iteration = 8 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
1962024-11-28T02:00:10.573ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
1972024-11-28T02:00:10.574ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
1982024-11-28T02:00:10.574ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1992024-11-28T02:00:10.574ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 96.451985ms iteration = 15 status = {"errors":[],"garbage_collect_requested":[]}
2002024-11-28T02:00:10.575ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2012024-11-28T02:00:10.575ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2022024-11-28T02:00:10.576ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
2032024-11-28T02:00:10.576ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
2042024-11-28T02:00:10.577ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
2052024-11-28T02:00:10.577ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
2062024-11-28T02:00:10.578ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
2072024-11-28T02:00:10.579ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
2082024-11-28T02:00:10.579ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
2092024-11-28T02:00:10.580ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
2102024-11-28T02:00:10.580ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 313847b3-d670-4167-9b09-9e7d3dfcb2a4, lookup_type: ById(313847b3-d670-4167-9b09-9e7d3dfcb2a4) } result = Ok(())
2112024-11-28T02:00:10.580ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
2122024-11-28T02:00:10.580ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
2132024-11-28T02:00:10.581ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
2142024-11-28T02:00:10.581ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
2152024-11-28T02:00:10.581ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
2162024-11-28T02:00:10.581ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
2172024-11-28T02:00:10.582ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
2182024-11-28T02:00:10.585ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
2192024-11-28T02:00:10.585ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
2202024-11-28T02:00:10.585ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
2212024-11-28T02:00:10.585ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
2222024-11-28T02:00:10.585ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
2232024-11-28T02:00:10.585ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 67ac247f-c934-4c78-857a-fba4fbeb9937 original_time_created = 2024-11-28 01:53:03.159475 UTC target_id = 67ac247f-c934-4c78-857a-fba4fbeb9937 time_created = 2024-11-28 01:53:03.159475 UTC
2242024-11-28T02:00:10.585ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 107.65893ms iteration = 15 status = 0
2252024-11-28T02:00:10.585ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 107.476167ms iteration = 43 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"67ac247f-c934-4c78-857a-fba4fbeb9937","time_created":"2024-11-28T01:53:03.159475Z"}
2262024-11-28T02:00:10.586ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
2272024-11-28T02:00:10.586ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
2282024-11-28T02:00:10.586ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-313847b3-d670-4167-9b09-9e7d3dfcb2a4.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-313847b3-d670-4167-9b09-9e7d3dfcb2a4.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1277, tv_nsec: 58123314 } })
2292024-11-28T02:00:10.586ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
2302024-11-28T02:00:10.587ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 108.524765ms iteration = 15 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
2312024-11-28T02:00:10.587ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2322024-11-28T02:00:10.587ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1103 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc
2332024-11-28T02:00:10.587ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2342024-11-28T02:00:10.587ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
2352024-11-28T02:00:10.587ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
2362024-11-28T02:00:10.587ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
2372024-11-28T02:00:10.588ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2382024-11-28T02:00:10.588ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
2392024-11-28T02:00:10.588ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
2402024-11-28T02:00:10.591ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
2412024-11-28T02:00:10.591ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
2422024-11-28T02:00:10.591ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
2432024-11-28T02:00:10.591ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
2442024-11-28T02:00:10.591ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
2452024-11-28T02:00:10.591ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
2462024-11-28T02:00:10.593ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
2472024-11-28T02:00:10.593ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
2482024-11-28T02:00:10.594ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
2492024-11-28T02:00:10.597ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
2502024-11-28T02:00:10.597ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
2512024-11-28T02:00:10.597ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
2522024-11-28T02:00:10.597ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
2532024-11-28T02:00:10.600ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
2542024-11-28T02:00:10.612ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
2552024-11-28T02:00:10.612ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
2562024-11-28T02:00:10.613ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
2572024-11-28T02:00:10.614ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
2582024-11-28T02:00:10.616ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2592024-11-28T02:00:10.619ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
2602024-11-28T02:00:10.619ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
2612024-11-28T02:00:10.619ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
2622024-11-28T02:00:10.624ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
2632024-11-28T02:00:10.624ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
2642024-11-28T02:00:10.624ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
2652024-11-28T02:00:10.624ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:140 found = 0 not_ready_to_be_deleted = 0
2662024-11-28T02:00:10.624ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 146.165279ms iteration = 8 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
2672024-11-28T02:00:10.624ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "fc9d1d38-afa0-4ade-91a9-1f7e271dea44", "content-length": "26", "date": "Thu, 28 Nov 2024 02:00:10 GMT"} })
2682024-11-28T02:00:10.624ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
2692024-11-28T02:00:10.625ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
2702024-11-28T02:00:10.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
2712024-11-28T02:00:10.626ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
2722024-11-28T02:00:10.626ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
2732024-11-28T02:00:10.626ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "0c87b668-175b-4bd7-9599-c79cc3a10942", "content-length": "26", "date": "Thu, 28 Nov 2024 02:00:10 GMT"} })
2742024-11-28T02:00:10.626ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
2752024-11-28T02:00:10.627ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
2762024-11-28T02:00:10.629ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
2772024-11-28T02:00:10.639ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 161.122789ms iteration = 15 status = {"errors":[],"requests_created_ok":[],"start_invoked_ok":[]}
2782024-11-28T02:00:10.642ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
2792024-11-28T02:00:10.642ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = a7e6af61-408c-483d-b043-c27ccf55708e dns_zone_name = oxide.test found_so_far = 0 version = 2
2802024-11-28T02:00:10.642ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
2812024-11-28T02:00:10.643ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
2822024-11-28T02:00:10.644ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2832024-11-28T02:00:10.646ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
2842024-11-28T02:00:10.646ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
2852024-11-28T02:00:10.646ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 168.009088ms iteration = 15 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
2862024-11-28T02:00:10.648ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
2872024-11-28T02:00:10.648ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 808a324e-c913-4bfc-91a5-d43c961cd511 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
2882024-11-28T02:00:10.648ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
2892024-11-28T02:00:10.649ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
2902024-11-28T02:00:10.649ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
2912024-11-28T02:00:10.649ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
2922024-11-28T02:00:10.649ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
2932024-11-28T02:00:10.650ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2942024-11-28T02:00:10.651ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = a7e6af61-408c-483d-b043-c27ccf55708e dns_zone_name = oxide.test found_so_far = 1 version = 2
2952024-11-28T02:00:10.651ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
2962024-11-28T02:00:10.651ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2024-11-28 01:53:43.351579 UTC dns_group = external generation = 2
2972024-11-28T02:00:10.651ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 173.549904ms iteration = 9 status = {"generation":2}
2982024-11-28T02:00:10.654ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 176.709689ms iteration = 15 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[]}
2992024-11-28T02:00:10.655ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
3002024-11-28T02:00:10.655ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
3012024-11-28T02:00:10.656ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
3022024-11-28T02:00:10.658ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 808a324e-c913-4bfc-91a5-d43c961cd511 dns_zone_name = control-plane.oxide.internal found_so_far = 51 version = 1
3032024-11-28T02:00:10.658ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
3042024-11-28T02:00:10.658ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2024-11-28 01:53:42.502562 UTC dns_group = internal generation = 1
3052024-11-28T02:00:10.658ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 180.969133ms iteration = 9 status = {"generation":1}
3062024-11-28T02:00:10.662ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
3072024-11-28T02:00:10.662ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
3082024-11-28T02:00:10.688ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2024-11-28T01:53:08.890632Z, time_modified: 2024-11-28T01:53:08.890632Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 7e159834-997a-4c03-b9db-1e91d783221c, db_silo: Silo { identity: SiloIdentity { id: 7e159834-997a-4c03-b9db-1e91d783221c, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-11-28T01:53:43.308696Z, time_modified: 2024-11-28T01:53:43.308696Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "bc39d1b3b4bb84a49d6458ed7aedc5bca9e77742c7b35882b64d188c0602d306" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 7e159834-997a-4c03-b9db-1e91d783221c, db_silo: Silo { identity: SiloIdentity { id: 7e159834-997a-4c03-b9db-1e91d783221c, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-11-28T01:53:43.308696Z, time_modified: 2024-11-28T01:53:43.308696Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "bc39d1b3b4bb84a49d6458ed7aedc5bca9e77742c7b35882b64d188c0602d306" }] }) }
3092024-11-28T02:00:10.689ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 210.564287ms iteration = 9 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"7e159834-997a-4c03-b9db-1e91d783221c","tls_certs":["bc39d1b3b4bb84a49d6458ed7aedc5bca9e77742c7b35882b64d188c0602d306"]}},"default_endpoint":{"silo_id":"7e159834-997a-4c03-b9db-1e91d783221c","tls_certs":["bc39d1b3b4bb84a49d6458ed7aedc5bca9e77742c7b35882b64d188c0602d306"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
3102024-11-28T02:00:10.762ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 284.446201ms iteration = 18 status = {"physical_disks_added":0}
3112024-11-28T02:00:10.788ZWARNnexus (BackgroundTasks): instance reincarnation completed with errors background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:103 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0 query_errors = 0 restart_errors = 0
3122024-11-28T02:00:10.788ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 309.967415ms iteration = 8 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
3132024-11-28T02:00:10.797ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
3142024-11-28T02:00:10.799ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
3152024-11-28T02:00:10.799ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "860c521b-c00d-46b8-995a-45c7bc072578", "content-length": "1", "date": "Thu, 28 Nov 2024 02:00:10 GMT"} })
3162024-11-28T02:00:10.802ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
3172024-11-28T02:00:10.802ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
3182024-11-28T02:00:10.824ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 345.949409ms iteration = 15 status = {}
3192024-11-28T02:00:10.829ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
3202024-11-28T02:00:10.829ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
3212024-11-28T02:00:10.888ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1111 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc zone_address = fd00:1122:3344:101::2
3222024-11-28T02:00:10.888ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
3232024-11-28T02:00:10.888ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "d6bf94a8-d644-40ff-9c84-a8d69b7260ed", "content-length": "26", "date": "Thu, 28 Nov 2024 02:00:10 GMT"} })
3242024-11-28T02:00:10.888ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1116 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
3252024-11-28T02:00:10.891ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc
3262024-11-28T02:00:10.940ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
3272024-11-28T02:00:10.941ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "5995ded8-6a4f-4ba8-836f-06ad1abeb061", "date": "Thu, 28 Nov 2024 02:00:10 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
3282024-11-28T02:00:10.941ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
3292024-11-28T02:00:10.944ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "068f8f80-cf71-40a0-9d37-caecc0cb0e4b", "date": "Thu, 28 Nov 2024 02:00:10 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
3302024-11-28T02:00:10.944ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
3312024-11-28T02:00:10.945ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
3322024-11-28T02:00:10.946ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 467.947028ms iteration = 10 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
3332024-11-28T02:00:10.964ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
3342024-11-28T02:00:10.964ZINFOnexus (BackgroundTasks): VPC route manager sled 313847b3-d670-4167-9b09-9e7d3dfcb2a4 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
3352024-11-28T02:00:10.964ZDEBGnexus (BackgroundTasks): client request SledAgent = 313847b3-d670-4167-9b09-9e7d3dfcb2a4 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
3362024-11-28T02:00:10.965ZDEBGnexus (BackgroundTasks): client response SledAgent = 313847b3-d670-4167-9b09-9e7d3dfcb2a4 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "f28e6a03-840c-4e81-976f-3816913a7cb8", "content-length": "372", "date": "Thu, 28 Nov 2024 02:00:10 GMT"} })
3372024-11-28T02:00:10.965ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
3382024-11-28T02:00:10.965ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "067f1768-3f33-46c8-ae90-ebd27f9b676d", "date": "Thu, 28 Nov 2024 02:00:10 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
3392024-11-28T02:00:10.965ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
3402024-11-28T02:00:10.965ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
3412024-11-28T02:00:10.967ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "63b3e975-c4dc-461e-a895-afb87dcf2799", "date": "Thu, 28 Nov 2024 02:00:10 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
3422024-11-28T02:00:10.967ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
3432024-11-28T02:00:10.967ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3442024-11-28T02:00:10.967ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 489.319269ms iteration = 15 status = null
3452024-11-28T02:00:10.970ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
3462024-11-28T02:00:10.970ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
3472024-11-28T02:00:10.970ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "55a720ee-22bb-4162-8c9c-a843b82d929d", "date": "Thu, 28 Nov 2024 02:00:10 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
3482024-11-28T02:00:10.970ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
3492024-11-28T02:00:10.970ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
3502024-11-28T02:00:10.972ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3512024-11-28T02:00:10.972ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
3522024-11-28T02:00:10.972ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 494.744424ms iteration = 10 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
3532024-11-28T02:00:10.974ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
3542024-11-28T02:00:10.974ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
3552024-11-28T02:00:10.974ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
3562024-11-28T02:00:11.024ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
3572024-11-28T02:00:11.024ZDEBGnexus (BackgroundTasks): client request SledAgent = 313847b3-d670-4167-9b09-9e7d3dfcb2a4 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
3582024-11-28T02:00:11.024ZDEBGnexus (BackgroundTasks): client response SledAgent = 313847b3-d670-4167-9b09-9e7d3dfcb2a4 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "eb04e11f-71d1-4497-81c1-955fc34e0890", "date": "Thu, 28 Nov 2024 02:00:10 GMT"} })
3592024-11-28T02:00:11.024ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
3602024-11-28T02:00:11.025ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
3612024-11-28T02:00:11.032ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
3622024-11-28T02:00:11.033ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
3632024-11-28T02:00:11.036ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
3642024-11-28T02:00:11.037ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
3652024-11-28T02:00:11.040ZINFOnexus (BackgroundTasks): VPC route manager sled 313847b3-d670-4167-9b09-9e7d3dfcb2a4 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
3662024-11-28T02:00:11.040ZDEBGnexus (BackgroundTasks): client request SledAgent = 313847b3-d670-4167-9b09-9e7d3dfcb2a4 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
3672024-11-28T02:00:11.059ZDEBGnexus (BackgroundTasks): client request SledAgent = 313847b3-d670-4167-9b09-9e7d3dfcb2a4 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
3682024-11-28T02:00:11.103ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
3692024-11-28T02:00:11.104ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
3702024-11-28T02:00:11.106ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 4bb2986f-2e92-4ac3-9dc1-591b6bb60b3e, rack_id: 92a8ff6e-21d3-4c28-bcad-e323847985dc, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(ea300318-9d62-42ed-80ea-21c86590a166) } rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc switch_location = Switch0
3712024-11-28T02:00:11.106ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
3722024-11-28T02:00:11.106ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
3732024-11-28T02:00:11.119ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
3742024-11-28T02:00:11.119ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "b5d1d6dc-292b-4c40-ac33-070bf2e8667f", "content-length": "134", "date": "Thu, 28 Nov 2024 02:00:11 GMT"} })
3752024-11-28T02:00:11.119ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1746 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
3762024-11-28T02:00:11.119ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1773 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = ea300318-9d62-42ed-80ea-21c86590a166
3772024-11-28T02:00:11.119ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
3782024-11-28T02:00:11.119ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
3792024-11-28T02:00:11.120ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
3802024-11-28T02:00:11.120ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
3812024-11-28T02:00:11.120ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
3822024-11-28T02:00:11.120ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1920 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
3832024-11-28T02:00:11.120ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
3842024-11-28T02:00:11.120ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
3852024-11-28T02:00:11.120ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc
3862024-11-28T02:00:11.120ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
3872024-11-28T02:00:11.120ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
3882024-11-28T02:00:11.122ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
3892024-11-28T02:00:11.123ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "85465e73-9d07-4e3c-b7fa-51adb7fc9730", "content-length": "2", "date": "Thu, 28 Nov 2024 02:00:11 GMT"} })
3902024-11-28T02:00:11.123ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
3912024-11-28T02:00:11.123ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "20671988-3d5d-461e-925d-dc973a024d9a", "content-length": "52", "date": "Thu, 28 Nov 2024 02:00:11 GMT"} })
3922024-11-28T02:00:11.123ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc switch_location = Switch0
3932024-11-28T02:00:11.123ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
3942024-11-28T02:00:11.142ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:40402 req_id = 65642ffa-2da8-4996-8aa0-59d456abeb08 roles = RoleSet { roles: {} } uri = /nat/ipv4/changeset/6?limit=100
3952024-11-28T02:00:11.143ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:40402 req_id = 65642ffa-2da8-4996-8aa0-59d456abeb08 resource = Database result = Ok(()) uri = /nat/ipv4/changeset/6?limit=100
3962024-11-28T02:00:11.146ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 3934 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:40402 req_id = 65642ffa-2da8-4996-8aa0-59d456abeb08 response_code = 200 uri = /nat/ipv4/changeset/6?limit=100
3972024-11-28T02:00:11.175ZDEBGnexus (BackgroundTasks): client response SledAgent = 313847b3-d670-4167-9b09-9e7d3dfcb2a4 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "095872ab-0c4c-4899-9c8e-807f87f95dc0", "date": "Thu, 28 Nov 2024 02:00:11 GMT"} })
3982024-11-28T02:00:11.178ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 700.09343ms iteration = 16 status = {}
3992024-11-28T02:00:11.178ZDEBGnexus (BackgroundTasks): client response SledAgent = 313847b3-d670-4167-9b09-9e7d3dfcb2a4 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "d1f017f9-a70a-4458-9bb1-adb881201e77", "content-length": "691", "date": "Thu, 28 Nov 2024 02:00:11 GMT"} })
4002024-11-28T02:00:11.178ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
4012024-11-28T02:00:11.178ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
4022024-11-28T02:00:11.178ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
4032024-11-28T02:00:11.180ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 702.270918ms iteration = 15 status = {}
4042024-11-28T02:00:11.365ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "18e5ecd9-e1ed-4daf-9d21-38c284aa4d74", "date": "Thu, 28 Nov 2024 02:00:10 GMT"} })
4052024-11-28T02:00:11.365ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc switch = Switch0
4062024-11-28T02:00:11.365ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
4072024-11-28T02:00:11.366ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "1a6a1b91-6a79-4e7d-a86b-6ce735fb0e11", "content-length": "682", "date": "Thu, 28 Nov 2024 02:00:11 GMT"} })
4082024-11-28T02:00:11.366ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
4092024-11-28T02:00:11.366ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
4102024-11-28T02:00:11.370ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
4112024-11-28T02:00:11.370ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
4122024-11-28T02:00:11.374ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
4132024-11-28T02:00:11.374ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
4142024-11-28T02:00:11.376ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
4152024-11-28T02:00:11.376ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
4162024-11-28T02:00:11.384ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 905.961845ms iteration = 15 status = {}
4172024-11-28T02:00:11.504ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 2033836d-1d69-4923-946b-92f69357d4ee roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
4182024-11-28T02:00:11.505ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 2033836d-1d69-4923-946b-92f69357d4ee resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
4192024-11-28T02:00:11.507ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2365 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 2033836d-1d69-4923-946b-92f69357d4ee response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
4202024-11-28T02:00:12.526ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = ec295955-9976-412d-be1d-9d245fe7f15c roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
4212024-11-28T02:00:12.526ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = ec295955-9976-412d-be1d-9d245fe7f15c resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
4222024-11-28T02:00:12.528ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2109 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = ec295955-9976-412d-be1d-9d245fe7f15c response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
4232024-11-28T02:00:13.545ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = d49a5630-36ce-4d19-821a-2c8ee791ab5b roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
4242024-11-28T02:00:13.546ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = d49a5630-36ce-4d19-821a-2c8ee791ab5b resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
4252024-11-28T02:00:13.548ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2173 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = d49a5630-36ce-4d19-821a-2c8ee791ab5b response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
4262024-11-28T02:00:14.194ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1355 local_addr = [fd00:1122:3344:101::b]:55621 method = GET remote_addr = [fd00:1122:3344:101::d]:56417 req_id = aa95c27c-4300-4906-9d9b-80c5a4e5992e response_code = 200 uri = /2469a748-bcd2-4e0d-99cc-f79def0452cc
4272024-11-28T02:00:14.566ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 973d7778-426f-461f-8cea-7a61b9596d37 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
4282024-11-28T02:00:14.566ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 973d7778-426f-461f-8cea-7a61b9596d37 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
4292024-11-28T02:00:14.568ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2564 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 973d7778-426f-461f-8cea-7a61b9596d37 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
4302024-11-28T02:00:15.588ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 82181f60-524c-413c-a197-2b5bf5a3bfb8 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
4312024-11-28T02:00:15.588ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 82181f60-524c-413c-a197-2b5bf5a3bfb8 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
4322024-11-28T02:00:15.590ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2020 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 82181f60-524c-413c-a197-2b5bf5a3bfb8 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
4332024-11-28T02:00:16.609ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 274a4985-a918-4620-ad90-a2df1636911a roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
4342024-11-28T02:00:16.609ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 274a4985-a918-4620-ad90-a2df1636911a resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
4352024-11-28T02:00:16.610ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1911 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 274a4985-a918-4620-ad90-a2df1636911a response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
4362024-11-28T02:00:17.630ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = cf9426a5-d56a-482d-b7af-fca6e635cc56 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
4372024-11-28T02:00:17.631ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = cf9426a5-d56a-482d-b7af-fca6e635cc56 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
4382024-11-28T02:00:17.633ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2426 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = cf9426a5-d56a-482d-b7af-fca6e635cc56 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
4392024-11-28T02:00:18.651ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 686b8c89-81d2-4edf-97cb-51b1e62d7a09 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
4402024-11-28T02:00:18.652ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 686b8c89-81d2-4edf-97cb-51b1e62d7a09 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
4412024-11-28T02:00:18.654ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2375 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 686b8c89-81d2-4edf-97cb-51b1e62d7a09 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
4422024-11-28T02:00:19.673ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 36ec7e0c-b924-490f-9483-76052a1154a2 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
4432024-11-28T02:00:19.673ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 36ec7e0c-b924-490f-9483-76052a1154a2 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
4442024-11-28T02:00:19.675ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2530 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 36ec7e0c-b924-490f-9483-76052a1154a2 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
4452024-11-28T02:00:20.477ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 44 reason = Timeout
4462024-11-28T02:00:20.478ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
4472024-11-28T02:00:20.478ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 44 reason = Timeout
4482024-11-28T02:00:20.478ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4492024-11-28T02:00:20.478ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
4502024-11-28T02:00:20.478ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
4512024-11-28T02:00:20.479ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4522024-11-28T02:00:20.480ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
4532024-11-28T02:00:20.480ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
4542024-11-28T02:00:20.482ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 4.548179ms iteration = 44 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
4552024-11-28T02:00:20.482ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
4562024-11-28T02:00:20.482ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4572024-11-28T02:00:20.482ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
4582024-11-28T02:00:20.483ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4592024-11-28T02:00:20.484ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
4602024-11-28T02:00:20.485ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4612024-11-28T02:00:20.488ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 67ac247f-c934-4c78-857a-fba4fbeb9937, lookup_type: ById(67ac247f-c934-4c78-857a-fba4fbeb9937) } result = Ok(())
4622024-11-28T02:00:20.488ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4632024-11-28T02:00:20.488ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
4642024-11-28T02:00:20.497ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 67ac247f-c934-4c78-857a-fba4fbeb9937 original_time_created = 2024-11-28 01:53:03.159475 UTC target_id = 67ac247f-c934-4c78-857a-fba4fbeb9937 time_created = 2024-11-28 01:53:03.159475 UTC
4652024-11-28T02:00:20.497ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 19.340096ms iteration = 44 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"67ac247f-c934-4c78-857a-fba4fbeb9937","time_created":"2024-11-28T01:53:03.159475Z"}
4662024-11-28T02:00:20.695ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = cfcbbc4b-ad1e-4044-b9f6-72821fcef166 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
4672024-11-28T02:00:20.695ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = cfcbbc4b-ad1e-4044-b9f6-72821fcef166 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
4682024-11-28T02:00:20.697ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2194 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = cfcbbc4b-ad1e-4044-b9f6-72821fcef166 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
4692024-11-28T02:00:21.716ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 6313feb1-ae38-4277-84f5-ebcbdc19a82a roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
4702024-11-28T02:00:21.716ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 6313feb1-ae38-4277-84f5-ebcbdc19a82a resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
4712024-11-28T02:00:21.718ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2005 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 6313feb1-ae38-4277-84f5-ebcbdc19a82a response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
4722024-11-28T02:00:22.736ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 67267f4c-779e-4399-983b-cad9d3ec3b4e roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
4732024-11-28T02:00:22.737ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 67267f4c-779e-4399-983b-cad9d3ec3b4e resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
4742024-11-28T02:00:22.738ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1870 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 67267f4c-779e-4399-983b-cad9d3ec3b4e response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
4752024-11-28T02:00:23.757ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = b8a43bc7-446e-47cf-857f-58a1738cd6c4 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
4762024-11-28T02:00:23.757ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = b8a43bc7-446e-47cf-857f-58a1738cd6c4 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
4772024-11-28T02:00:23.759ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2103 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = b8a43bc7-446e-47cf-857f-58a1738cd6c4 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
4782024-11-28T02:00:24.193ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1661 local_addr = [fd00:1122:3344:101::b]:55621 method = GET remote_addr = [fd00:1122:3344:101::d]:56417 req_id = 9c1b52cf-9430-4a28-bcad-7a734b12ddee response_code = 200 uri = /2469a748-bcd2-4e0d-99cc-f79def0452cc
4792024-11-28T02:00:24.778ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 39a33c39-0bda-4292-b0d2-7be3088ab778 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
4802024-11-28T02:00:24.778ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 39a33c39-0bda-4292-b0d2-7be3088ab778 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
4812024-11-28T02:00:24.779ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1657 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 39a33c39-0bda-4292-b0d2-7be3088ab778 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
4822024-11-28T02:00:25.799ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 97adc88d-4584-4822-8195-67d1e5932e42 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
4832024-11-28T02:00:25.799ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 97adc88d-4584-4822-8195-67d1e5932e42 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
4842024-11-28T02:00:25.801ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2428 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 97adc88d-4584-4822-8195-67d1e5932e42 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
4852024-11-28T02:00:26.821ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 2fa610c6-6632-4013-b320-43250ab142be roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
4862024-11-28T02:00:26.821ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 2fa610c6-6632-4013-b320-43250ab142be resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
4872024-11-28T02:00:26.823ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2330 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 2fa610c6-6632-4013-b320-43250ab142be response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
4882024-11-28T02:00:27.841ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 2164bbfe-45d2-4281-9d27-bc0f9c4e0e6e roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
4892024-11-28T02:00:27.842ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 2164bbfe-45d2-4281-9d27-bc0f9c4e0e6e resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
4902024-11-28T02:00:27.843ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1571 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 2164bbfe-45d2-4281-9d27-bc0f9c4e0e6e response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
4912024-11-28T02:00:28.862ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 5bd6f590-ffd8-4529-b4a5-3dae0f53f014 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
4922024-11-28T02:00:28.863ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 5bd6f590-ffd8-4529-b4a5-3dae0f53f014 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
4932024-11-28T02:00:28.865ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2347 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 5bd6f590-ffd8-4529-b4a5-3dae0f53f014 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
4942024-11-28T02:00:29.882ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 73f30449-b261-4369-9aa7-a6b7fd64ff62 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
4952024-11-28T02:00:29.883ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 73f30449-b261-4369-9aa7-a6b7fd64ff62 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
4962024-11-28T02:00:29.885ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2158 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 73f30449-b261-4369-9aa7-a6b7fd64ff62 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
4972024-11-28T02:00:30.478ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 45 reason = Timeout
4982024-11-28T02:00:30.478ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
4992024-11-28T02:00:30.478ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 45 reason = Timeout
5002024-11-28T02:00:30.478ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5012024-11-28T02:00:30.478ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
5022024-11-28T02:00:30.478ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5032024-11-28T02:00:30.480ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
5042024-11-28T02:00:30.480ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
5052024-11-28T02:00:30.480ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5062024-11-28T02:00:30.483ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
5072024-11-28T02:00:30.483ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5082024-11-28T02:00:30.483ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5092024-11-28T02:00:30.485ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5102024-11-28T02:00:30.485ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5112024-11-28T02:00:30.486ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5122024-11-28T02:00:30.489ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 67ac247f-c934-4c78-857a-fba4fbeb9937, lookup_type: ById(67ac247f-c934-4c78-857a-fba4fbeb9937) } result = Ok(())
5132024-11-28T02:00:30.489ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5142024-11-28T02:00:30.489ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5152024-11-28T02:00:30.501ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 67ac247f-c934-4c78-857a-fba4fbeb9937 original_time_created = 2024-11-28 01:53:03.159475 UTC target_id = 67ac247f-c934-4c78-857a-fba4fbeb9937 time_created = 2024-11-28 01:53:03.159475 UTC
5162024-11-28T02:00:30.501ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 22.906838ms iteration = 45 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"67ac247f-c934-4c78-857a-fba4fbeb9937","time_created":"2024-11-28T01:53:03.159475Z"}
5172024-11-28T02:00:30.515ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 36.993213ms iteration = 45 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
5182024-11-28T02:00:30.903ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = eac4ef36-173d-41f5-acbc-cfe0d65f036e roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
5192024-11-28T02:00:30.904ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = eac4ef36-173d-41f5-acbc-cfe0d65f036e resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
5202024-11-28T02:00:30.905ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2262 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = eac4ef36-173d-41f5-acbc-cfe0d65f036e response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
5212024-11-28T02:00:31.925ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 3f8c6373-bf20-4f1b-a712-0e78920e082b roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
5222024-11-28T02:00:31.925ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 3f8c6373-bf20-4f1b-a712-0e78920e082b resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
5232024-11-28T02:00:31.927ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2080 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 3f8c6373-bf20-4f1b-a712-0e78920e082b response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
5242024-11-28T02:00:32.947ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = a03bbaeb-a753-454b-9e5a-a7a4d0e38937 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
5252024-11-28T02:00:32.948ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = a03bbaeb-a753-454b-9e5a-a7a4d0e38937 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
5262024-11-28T02:00:32.949ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2046 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = a03bbaeb-a753-454b-9e5a-a7a4d0e38937 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
5272024-11-28T02:00:33.969ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 1f2c6f44-28d9-4409-9a59-4e09b802c186 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
5282024-11-28T02:00:33.969ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 1f2c6f44-28d9-4409-9a59-4e09b802c186 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
5292024-11-28T02:00:33.971ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1900 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 1f2c6f44-28d9-4409-9a59-4e09b802c186 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
5302024-11-28T02:00:34.193ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1811 local_addr = [fd00:1122:3344:101::b]:55621 method = GET remote_addr = [fd00:1122:3344:101::d]:56417 req_id = 40a65b93-d488-4af8-9818-92de20f19009 response_code = 200 uri = /2469a748-bcd2-4e0d-99cc-f79def0452cc
5312024-11-28T02:00:34.990ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = ff7c0b9e-320f-45e3-85a0-3e3159cd124e roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
5322024-11-28T02:00:34.990ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = ff7c0b9e-320f-45e3-85a0-3e3159cd124e resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
5332024-11-28T02:00:34.992ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1967 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = ff7c0b9e-320f-45e3-85a0-3e3159cd124e response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
5342024-11-28T02:00:36.011ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = f4c5b22b-384a-4b76-91b4-d0c929d04fe3 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
5352024-11-28T02:00:36.011ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = f4c5b22b-384a-4b76-91b4-d0c929d04fe3 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
5362024-11-28T02:00:36.013ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2365 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = f4c5b22b-384a-4b76-91b4-d0c929d04fe3 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
5372024-11-28T02:00:37.031ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 9ce221ac-c2e2-48a1-82ad-a1bc2569923d roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
5382024-11-28T02:00:37.031ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 9ce221ac-c2e2-48a1-82ad-a1bc2569923d resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
5392024-11-28T02:00:37.033ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1853 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 9ce221ac-c2e2-48a1-82ad-a1bc2569923d response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
5402024-11-28T02:00:38.052ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 2e9327f6-a987-41f1-9bd1-e8da4f12a11c roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
5412024-11-28T02:00:38.052ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 2e9327f6-a987-41f1-9bd1-e8da4f12a11c resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
5422024-11-28T02:00:38.054ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2754 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 2e9327f6-a987-41f1-9bd1-e8da4f12a11c response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
5432024-11-28T02:00:39.073ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 2c9a2f92-29c2-4c83-94fe-1a1e03bbbf51 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
5442024-11-28T02:00:39.073ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 2c9a2f92-29c2-4c83-94fe-1a1e03bbbf51 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
5452024-11-28T02:00:39.075ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2392 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 2c9a2f92-29c2-4c83-94fe-1a1e03bbbf51 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
5462024-11-28T02:00:39.185ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:55495 req_id = fa567ab1-90fd-4b14-958f-737962faa45a roles = RoleSet { roles: {} } uri = /metrics/collectors/119eed64-2e24-4d5f-8c42-be6cd8bec8b5/producers?sort_by=id_ascending
5472024-11-28T02:00:39.185ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:55495 req_id = fa567ab1-90fd-4b14-958f-737962faa45a resource = Database result = Ok(()) uri = /metrics/collectors/119eed64-2e24-4d5f-8c42-be6cd8bec8b5/producers?sort_by=id_ascending
5482024-11-28T02:00:39.188ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2888 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:55495 req_id = fa567ab1-90fd-4b14-958f-737962faa45a response_code = 200 uri = /metrics/collectors/119eed64-2e24-4d5f-8c42-be6cd8bec8b5/producers?sort_by=id_ascending
5492024-11-28T02:00:39.188ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:55495 req_id = d273ad36-7607-477b-95bc-d56d25c1374c roles = RoleSet { roles: {} } uri = /metrics/collectors/119eed64-2e24-4d5f-8c42-be6cd8bec8b5/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJjNDM0YjQ0OC01MTk5LTQ2NmUtYWYzMi00ZjYyOWUzMWM3YjEifX0%3D
5502024-11-28T02:00:39.188ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:55495 req_id = d273ad36-7607-477b-95bc-d56d25c1374c resource = Database result = Ok(()) uri = /metrics/collectors/119eed64-2e24-4d5f-8c42-be6cd8bec8b5/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJjNDM0YjQ0OC01MTk5LTQ2NmUtYWYzMi00ZjYyOWUzMWM3YjEifX0%3D
5512024-11-28T02:00:39.190ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1925 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:55495 req_id = d273ad36-7607-477b-95bc-d56d25c1374c response_code = 200 uri = /metrics/collectors/119eed64-2e24-4d5f-8c42-be6cd8bec8b5/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJjNDM0YjQ0OC01MTk5LTQ2NmUtYWYzMi00ZjYyOWUzMWM3YjEifX0%3D
5522024-11-28T02:00:39.835ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::b]:12221 remote_addr = [fd00:1122:3344:101::1]:60019
5532024-11-28T02:00:39.835ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::1]:60019 req_id = 62a7ab66-d2a1-4b6a-a574-e6e90a4e4581 roles = RoleSet { roles: {} } uri = /metrics/producers
5542024-11-28T02:00:39.836ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::1]:60019 req_id = 62a7ab66-d2a1-4b6a-a574-e6e90a4e4581 resource = Database result = Ok(()) uri = /metrics/producers
5552024-11-28T02:00:39.873ZINFOnexus: assigned collector to new producer collector_id = 119eed64-2e24-4d5f-8c42-be6cd8bec8b5 file = nexus/src/app/oximeter.rs:76 producer_id = 313847b3-d670-4167-9b09-9e7d3dfcb2a4
5562024-11-28T02:00:39.873ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 37675 local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::1]:60019 req_id = 62a7ab66-d2a1-4b6a-a574-e6e90a4e4581 response_code = 201 uri = /metrics/producers
5572024-11-28T02:00:40.094ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 6fda9c1a-6a14-42e9-bf1f-d086de4fc0fd roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
5582024-11-28T02:00:40.095ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 6fda9c1a-6a14-42e9-bf1f-d086de4fc0fd resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
5592024-11-28T02:00:40.097ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2287 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 6fda9c1a-6a14-42e9-bf1f-d086de4fc0fd response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
5602024-11-28T02:00:40.478ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 46 reason = Timeout
5612024-11-28T02:00:40.478ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
5622024-11-28T02:00:40.478ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 16 reason = Timeout
5632024-11-28T02:00:40.478ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
5642024-11-28T02:00:40.478ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 16 reason = Timeout
5652024-11-28T02:00:40.478ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 17 reason = Timeout
5662024-11-28T02:00:40.478ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
5672024-11-28T02:00:40.478ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
5682024-11-28T02:00:40.478ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
5692024-11-28T02:00:40.478ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 16 reason = Timeout
5702024-11-28T02:00:40.478ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 16 reason = Timeout
5712024-11-28T02:00:40.478ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 46 reason = Timeout
5722024-11-28T02:00:40.478ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
5732024-11-28T02:00:40.478ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
5742024-11-28T02:00:40.478ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 16 reason = Timeout
5752024-11-28T02:00:40.478ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5762024-11-28T02:00:40.478ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 16 reason = Timeout
5772024-11-28T02:00:40.478ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
5782024-11-28T02:00:40.478ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
5792024-11-28T02:00:40.478ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 16 reason = Timeout
5802024-11-28T02:00:40.478ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 16 reason = Timeout
5812024-11-28T02:00:40.478ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
5822024-11-28T02:00:40.478ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 16 reason = Timeout
5832024-11-28T02:00:40.478ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
5842024-11-28T02:00:40.478ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
5852024-11-28T02:00:40.478ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 16 reason = Timeout
5862024-11-28T02:00:40.478ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
5872024-11-28T02:00:40.478ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 16 reason = Timeout
5882024-11-28T02:00:40.478ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
5892024-11-28T02:00:40.478ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 16 reason = Timeout
5902024-11-28T02:00:40.478ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
5912024-11-28T02:00:40.478ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 16 reason = Timeout
5922024-11-28T02:00:40.478ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
5932024-11-28T02:00:40.478ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 19 reason = Timeout
5942024-11-28T02:00:40.478ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
5952024-11-28T02:00:40.478ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
5962024-11-28T02:00:40.478ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
5972024-11-28T02:00:40.478ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
5982024-11-28T02:00:40.478ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
5992024-11-28T02:00:40.478ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
6002024-11-28T02:00:40.478ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
6012024-11-28T02:00:40.478ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
6022024-11-28T02:00:40.478ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
6032024-11-28T02:00:40.478ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
6042024-11-28T02:00:40.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
6052024-11-28T02:00:40.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
6062024-11-28T02:00:40.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
6072024-11-28T02:00:40.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
6082024-11-28T02:00:40.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
6092024-11-28T02:00:40.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
6102024-11-28T02:00:40.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6112024-11-28T02:00:40.481ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
6122024-11-28T02:00:40.481ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
6132024-11-28T02:00:40.482ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 4.418687ms iteration = 16 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
6142024-11-28T02:00:40.483ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6152024-11-28T02:00:40.486ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
6162024-11-28T02:00:40.486ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
6172024-11-28T02:00:40.486ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
6182024-11-28T02:00:40.488ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6192024-11-28T02:00:40.489ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6202024-11-28T02:00:40.489ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 11.139213ms iteration = 16 status = {"errors":[],"records_set_to_done":[]}
6212024-11-28T02:00:40.491ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
6222024-11-28T02:00:40.491ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
6232024-11-28T02:00:40.491ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
6242024-11-28T02:00:40.492ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
6252024-11-28T02:00:40.492ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
6262024-11-28T02:00:40.492ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
6272024-11-28T02:00:40.494ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-313847b3-d670-4167-9b09-9e7d3dfcb2a4.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-313847b3-d670-4167-9b09-9e7d3dfcb2a4.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1306, tv_nsec: 965697858 } })
6282024-11-28T02:00:40.494ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-313847b3-d670-4167-9b09-9e7d3dfcb2a4.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-313847b3-d670-4167-9b09-9e7d3dfcb2a4.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1306, tv_nsec: 965974083 } })
6292024-11-28T02:00:40.494ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1103
6302024-11-28T02:00:40.495ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6312024-11-28T02:00:40.495ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1103
6322024-11-28T02:00:40.495ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
6332024-11-28T02:00:40.496ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
6342024-11-28T02:00:40.497ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
6352024-11-28T02:00:40.497ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
6362024-11-28T02:00:40.497ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
6372024-11-28T02:00:40.497ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6382024-11-28T02:00:40.498ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6392024-11-28T02:00:40.500ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-313847b3-d670-4167-9b09-9e7d3dfcb2a4.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-313847b3-d670-4167-9b09-9e7d3dfcb2a4.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1306, tv_nsec: 971284866 } })
6402024-11-28T02:00:40.500ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6412024-11-28T02:00:40.500ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1103 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc
6422024-11-28T02:00:40.501ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6432024-11-28T02:00:40.501ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6442024-11-28T02:00:40.503ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
6452024-11-28T02:00:40.503ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
6462024-11-28T02:00:40.503ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
6472024-11-28T02:00:40.514ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 36.502144ms iteration = 16 status = {"errors":[],"garbage_collect_requested":[]}
6482024-11-28T02:00:40.516ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6492024-11-28T02:00:40.519ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
6502024-11-28T02:00:40.519ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
6512024-11-28T02:00:40.519ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
6522024-11-28T02:00:40.523ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
6532024-11-28T02:00:40.524ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
6542024-11-28T02:00:40.525ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 47.409323ms iteration = 46 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
6552024-11-28T02:00:40.527ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
6562024-11-28T02:00:40.527ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
6572024-11-28T02:00:40.528ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6582024-11-28T02:00:40.529ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
6592024-11-28T02:00:40.529ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
6602024-11-28T02:00:40.530ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
6612024-11-28T02:00:40.530ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
6622024-11-28T02:00:40.531ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
6632024-11-28T02:00:40.531ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
6642024-11-28T02:00:40.532ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
6652024-11-28T02:00:40.532ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 53.729503ms iteration = 16 status = {"errors":[],"requests_created_ok":[],"start_invoked_ok":[]}
6662024-11-28T02:00:40.533ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
6672024-11-28T02:00:40.533ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
6682024-11-28T02:00:40.533ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
6692024-11-28T02:00:40.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
6702024-11-28T02:00:40.534ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6712024-11-28T02:00:40.534ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
6722024-11-28T02:00:40.534ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
6732024-11-28T02:00:40.536ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
6742024-11-28T02:00:40.536ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
6752024-11-28T02:00:40.536ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
6762024-11-28T02:00:40.543ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6772024-11-28T02:00:40.546ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 67ac247f-c934-4c78-857a-fba4fbeb9937, lookup_type: ById(67ac247f-c934-4c78-857a-fba4fbeb9937) } result = Ok(())
6782024-11-28T02:00:40.546ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6792024-11-28T02:00:40.546ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6802024-11-28T02:00:40.550ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
6812024-11-28T02:00:40.551ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
6822024-11-28T02:00:40.563ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
6832024-11-28T02:00:40.563ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 85.292502ms iteration = 16 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
6842024-11-28T02:00:40.568ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
6852024-11-28T02:00:40.568ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
6862024-11-28T02:00:40.571ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 92.852532ms iteration = 16 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[]}
6872024-11-28T02:00:40.573ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 67ac247f-c934-4c78-857a-fba4fbeb9937 original_time_created = 2024-11-28 01:53:03.159475 UTC target_id = 67ac247f-c934-4c78-857a-fba4fbeb9937 time_created = 2024-11-28 01:53:03.159475 UTC
6882024-11-28T02:00:40.573ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 94.678404ms iteration = 46 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"67ac247f-c934-4c78-857a-fba4fbeb9937","time_created":"2024-11-28T01:53:03.159475Z"}
6892024-11-28T02:00:40.576ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 98.201556ms iteration = 16 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
6902024-11-28T02:00:40.580ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
6912024-11-28T02:00:40.580ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
6922024-11-28T02:00:40.580ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
6932024-11-28T02:00:40.580ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
6942024-11-28T02:00:40.599ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
6952024-11-28T02:00:40.599ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
6962024-11-28T02:00:40.599ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "4ccc8703-6af3-4cee-a601-c304cc796793", "content-length": "26", "date": "Thu, 28 Nov 2024 02:00:40 GMT"} })
6972024-11-28T02:00:40.599ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
6982024-11-28T02:00:40.601ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
6992024-11-28T02:00:40.604ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
7002024-11-28T02:00:40.604ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
7012024-11-28T02:00:40.604ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "5395ad72-4673-43c2-94bf-78417ef16a7d", "content-length": "26", "date": "Thu, 28 Nov 2024 02:00:40 GMT"} })
7022024-11-28T02:00:40.604ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
7032024-11-28T02:00:40.606ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
7042024-11-28T02:00:40.612ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1111 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc zone_address = fd00:1122:3344:101::2
7052024-11-28T02:00:40.612ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
7062024-11-28T02:00:40.612ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "6a527b9a-e4b3-4a98-bd63-08442d03566a", "content-length": "26", "date": "Thu, 28 Nov 2024 02:00:40 GMT"} })
7072024-11-28T02:00:40.612ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1116 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
7082024-11-28T02:00:40.614ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc
7092024-11-28T02:00:40.620ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 142.037127ms iteration = 19 status = {"physical_disks_added":0}
7102024-11-28T02:00:40.657ZINFOnexus (BackgroundTasks): VPC route manager sled 313847b3-d670-4167-9b09-9e7d3dfcb2a4 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
7112024-11-28T02:00:40.657ZDEBGnexus (BackgroundTasks): client request SledAgent = 313847b3-d670-4167-9b09-9e7d3dfcb2a4 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
7122024-11-28T02:00:40.657ZDEBGnexus (BackgroundTasks): client response SledAgent = 313847b3-d670-4167-9b09-9e7d3dfcb2a4 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "fa7d5e9e-f7e1-4b9c-9585-11c0a3d4e892", "content-length": "372", "date": "Thu, 28 Nov 2024 02:00:40 GMT"} })
7132024-11-28T02:00:40.657ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
7142024-11-28T02:00:40.657ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
7152024-11-28T02:00:40.659ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7162024-11-28T02:00:40.660ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
7172024-11-28T02:00:40.660ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
7182024-11-28T02:00:40.661ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
7192024-11-28T02:00:40.661ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
7202024-11-28T02:00:40.661ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7212024-11-28T02:00:40.661ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
7222024-11-28T02:00:40.664ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 313847b3-d670-4167-9b09-9e7d3dfcb2a4, lookup_type: ById(313847b3-d670-4167-9b09-9e7d3dfcb2a4) } result = Ok(())
7232024-11-28T02:00:40.664ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
7242024-11-28T02:00:40.664ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
7252024-11-28T02:00:40.666ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 187.774882ms iteration = 16 status = 0
7262024-11-28T02:00:40.699ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
7272024-11-28T02:00:40.699ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
7282024-11-28T02:00:40.699ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 221.353605ms iteration = 16 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
7292024-11-28T02:00:40.701ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7302024-11-28T02:00:40.703ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
7312024-11-28T02:00:40.703ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
7322024-11-28T02:00:40.703ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
7332024-11-28T02:00:40.709ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 231.267827ms iteration = 16 status = {}
7342024-11-28T02:00:40.711ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
7352024-11-28T02:00:40.712ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "e4f29ae1-8e5f-4dba-aed9-4e294b897c0e", "content-length": "1", "date": "Thu, 28 Nov 2024 02:00:40 GMT"} })
7362024-11-28T02:00:40.714ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
7372024-11-28T02:00:40.714ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
7382024-11-28T02:00:40.716ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 237.837661ms iteration = 16 status = null
7392024-11-28T02:00:40.724ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
7402024-11-28T02:00:40.724ZDEBGnexus (BackgroundTasks): client request SledAgent = 313847b3-d670-4167-9b09-9e7d3dfcb2a4 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
7412024-11-28T02:00:40.724ZDEBGnexus (BackgroundTasks): client response SledAgent = 313847b3-d670-4167-9b09-9e7d3dfcb2a4 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "a0b92dfc-cbc1-4b38-9bf3-0409af4c5782", "date": "Thu, 28 Nov 2024 02:00:40 GMT"} })
7422024-11-28T02:00:40.724ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
7432024-11-28T02:00:40.725ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
7442024-11-28T02:00:40.733ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
7452024-11-28T02:00:40.734ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
7462024-11-28T02:00:40.738ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
7472024-11-28T02:00:40.739ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
7482024-11-28T02:00:40.742ZINFOnexus (BackgroundTasks): VPC route manager sled 313847b3-d670-4167-9b09-9e7d3dfcb2a4 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
7492024-11-28T02:00:40.742ZDEBGnexus (BackgroundTasks): client request SledAgent = 313847b3-d670-4167-9b09-9e7d3dfcb2a4 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
7502024-11-28T02:00:40.742ZDEBGnexus (BackgroundTasks): client response SledAgent = 313847b3-d670-4167-9b09-9e7d3dfcb2a4 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "908305c3-b777-4992-8d58-4a92d7e27a2e", "date": "Thu, 28 Nov 2024 02:00:40 GMT"} })
7512024-11-28T02:00:40.744ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 266.501679ms iteration = 17 status = {}
7522024-11-28T02:00:40.837ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
7532024-11-28T02:00:40.838ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
7542024-11-28T02:00:40.839ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 4bb2986f-2e92-4ac3-9dc1-591b6bb60b3e, rack_id: 92a8ff6e-21d3-4c28-bcad-e323847985dc, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(ea300318-9d62-42ed-80ea-21c86590a166) } rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc switch_location = Switch0
7552024-11-28T02:00:40.839ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
7562024-11-28T02:00:40.840ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
7572024-11-28T02:00:40.841ZDEBGnexus (BackgroundTasks): client request SledAgent = 313847b3-d670-4167-9b09-9e7d3dfcb2a4 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
7582024-11-28T02:00:40.842ZDEBGnexus (BackgroundTasks): client response SledAgent = 313847b3-d670-4167-9b09-9e7d3dfcb2a4 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "accaebdb-1ea3-4c1c-8954-6793e5e74cd9", "content-length": "691", "date": "Thu, 28 Nov 2024 02:00:40 GMT"} })
7592024-11-28T02:00:40.842ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
7602024-11-28T02:00:40.842ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
7612024-11-28T02:00:40.842ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
7622024-11-28T02:00:40.843ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 365.108132ms iteration = 16 status = {}
7632024-11-28T02:00:40.858ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
7642024-11-28T02:00:40.859ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "ce695a86-1b82-4d5b-b532-fc45d6fd4c76", "content-length": "134", "date": "Thu, 28 Nov 2024 02:00:40 GMT"} })
7652024-11-28T02:00:40.859ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1746 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
7662024-11-28T02:00:40.859ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1773 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = ea300318-9d62-42ed-80ea-21c86590a166
7672024-11-28T02:00:40.859ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
7682024-11-28T02:00:40.859ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
7692024-11-28T02:00:40.859ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
7702024-11-28T02:00:40.859ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
7712024-11-28T02:00:40.859ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
7722024-11-28T02:00:40.859ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1920 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
7732024-11-28T02:00:40.859ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
7742024-11-28T02:00:40.859ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
7752024-11-28T02:00:40.859ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc
7762024-11-28T02:00:40.859ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
7772024-11-28T02:00:40.860ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
7782024-11-28T02:00:40.862ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
7792024-11-28T02:00:40.862ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "60b271f9-7441-49e6-9f9e-811d85d7dfff", "content-length": "2", "date": "Thu, 28 Nov 2024 02:00:40 GMT"} })
7802024-11-28T02:00:40.862ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
7812024-11-28T02:00:40.862ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "96ef7040-99e4-450a-ad8c-9e260c6db3dc", "content-length": "52", "date": "Thu, 28 Nov 2024 02:00:40 GMT"} })
7822024-11-28T02:00:40.862ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc switch_location = Switch0
7832024-11-28T02:00:40.862ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
7842024-11-28T02:00:41.058ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "48a8e802-072d-4e79-b93c-68165e2d4282", "date": "Thu, 28 Nov 2024 02:00:40 GMT"} })
7852024-11-28T02:00:41.058ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc switch = Switch0
7862024-11-28T02:00:41.058ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
7872024-11-28T02:00:41.058ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "ad7989a5-e98d-4492-ab4a-8b6bd4e352c3", "content-length": "682", "date": "Thu, 28 Nov 2024 02:00:40 GMT"} })
7882024-11-28T02:00:41.058ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
7892024-11-28T02:00:41.059ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
7902024-11-28T02:00:41.062ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
7912024-11-28T02:00:41.062ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
7922024-11-28T02:00:41.066ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
7932024-11-28T02:00:41.066ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
7942024-11-28T02:00:41.067ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
7952024-11-28T02:00:41.068ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
7962024-11-28T02:00:41.075ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 596.938349ms iteration = 16 status = {}
7972024-11-28T02:00:41.178ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 32e66946-5c69-4b87-a786-7f387d568eb5 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
7982024-11-28T02:00:41.178ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 32e66946-5c69-4b87-a786-7f387d568eb5 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
7992024-11-28T02:00:41.178ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:40402 req_id = af87c82f-d0cb-4de5-a85f-11cbf06db507 roles = RoleSet { roles: {} } uri = /nat/ipv4/changeset/6?limit=100
8002024-11-28T02:00:41.178ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:40402 req_id = af87c82f-d0cb-4de5-a85f-11cbf06db507 resource = Database result = Ok(()) uri = /nat/ipv4/changeset/6?limit=100
8012024-11-28T02:00:41.180ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2445 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 32e66946-5c69-4b87-a786-7f387d568eb5 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
8022024-11-28T02:00:41.182ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 3778 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:40402 req_id = af87c82f-d0cb-4de5-a85f-11cbf06db507 response_code = 200 uri = /nat/ipv4/changeset/6?limit=100
8032024-11-28T02:00:41.765ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::b]:12221 remote_addr = [fd00:1122:3344:101::2]:54776
8042024-11-28T02:00:41.765ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:54776 req_id = 05b43ac7-2ebc-48fa-b84a-4513af4707f2 roles = RoleSet { roles: {} } uri = /metrics/producers
8052024-11-28T02:00:41.766ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:54776 req_id = 05b43ac7-2ebc-48fa-b84a-4513af4707f2 resource = Database result = Ok(()) uri = /metrics/producers
8062024-11-28T02:00:41.805ZINFOnexus: assigned collector to new producer collector_id = 119eed64-2e24-4d5f-8c42-be6cd8bec8b5 file = nexus/src/app/oximeter.rs:76 producer_id = 4ea85f4f-c0ac-4d9e-ae1a-e4f649a8809c
8072024-11-28T02:00:41.805ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 39743 local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:54776 req_id = 05b43ac7-2ebc-48fa-b84a-4513af4707f2 response_code = 201 uri = /metrics/producers
8082024-11-28T02:00:41.965ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::b]:12221 remote_addr = [fd00:1122:3344:101::1]:37735
8092024-11-28T02:00:41.966ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::1]:37735 req_id = b814dc34-6e65-451e-b8fe-715d728f8800 roles = RoleSet { roles: {} } uri = /metrics/producers
8102024-11-28T02:00:41.966ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::1]:37735 req_id = b814dc34-6e65-451e-b8fe-715d728f8800 resource = Database result = Ok(()) uri = /metrics/producers
8112024-11-28T02:00:42.007ZINFOnexus: assigned collector to new producer collector_id = 119eed64-2e24-4d5f-8c42-be6cd8bec8b5 file = nexus/src/app/oximeter.rs:76 producer_id = 7d8f995c-f158-4cd9-9fae-a9ac17bfe5ed
8122024-11-28T02:00:42.007ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 41545 local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::1]:37735 req_id = b814dc34-6e65-451e-b8fe-715d728f8800 response_code = 201 uri = /metrics/producers
8132024-11-28T02:00:42.199ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 7ff9bc6a-dee8-4861-a2c5-e104463dadf2 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
8142024-11-28T02:00:42.199ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 7ff9bc6a-dee8-4861-a2c5-e104463dadf2 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
8152024-11-28T02:00:42.201ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2134 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 7ff9bc6a-dee8-4861-a2c5-e104463dadf2 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
8162024-11-28T02:00:42.562ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::b]:12221 remote_addr = [fd00:1122:3344:101::2]:62454
8172024-11-28T02:00:42.562ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:62454 req_id = 498b6b7d-b6f1-4272-afb9-befbf0fbd551 roles = RoleSet { roles: {} } uri = /metrics/producers
8182024-11-28T02:00:42.563ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:62454 req_id = 498b6b7d-b6f1-4272-afb9-befbf0fbd551 resource = Database result = Ok(()) uri = /metrics/producers
8192024-11-28T02:00:42.601ZINFOnexus: assigned collector to new producer collector_id = 119eed64-2e24-4d5f-8c42-be6cd8bec8b5 file = nexus/src/app/oximeter.rs:76 producer_id = 37e16e5f-8832-402e-b7dd-2f5b87405438
8202024-11-28T02:00:42.601ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 38510 local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:62454 req_id = 498b6b7d-b6f1-4272-afb9-befbf0fbd551 response_code = 201 uri = /metrics/producers
8212024-11-28T02:00:43.220ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = f1a84294-ff0b-4ea0-9f6e-a00b31dca583 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
8222024-11-28T02:00:43.221ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = f1a84294-ff0b-4ea0-9f6e-a00b31dca583 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
8232024-11-28T02:00:43.223ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2901 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = f1a84294-ff0b-4ea0-9f6e-a00b31dca583 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
8242024-11-28T02:00:44.194ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1210 local_addr = [fd00:1122:3344:101::b]:55621 method = GET remote_addr = [fd00:1122:3344:101::d]:56417 req_id = 26fee773-d5be-4e09-a775-7d5ab7939efd response_code = 200 uri = /2469a748-bcd2-4e0d-99cc-f79def0452cc
8252024-11-28T02:00:44.241ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = d4b8c8d7-28d4-44a6-a122-7536c5ad36eb roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
8262024-11-28T02:00:44.241ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = d4b8c8d7-28d4-44a6-a122-7536c5ad36eb resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
8272024-11-28T02:00:44.243ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2253 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = d4b8c8d7-28d4-44a6-a122-7536c5ad36eb response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
8282024-11-28T02:00:45.261ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = d4d434a3-0fef-424e-831a-1465d3896415 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
8292024-11-28T02:00:45.262ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = d4d434a3-0fef-424e-831a-1465d3896415 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
8302024-11-28T02:00:45.264ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2238 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = d4d434a3-0fef-424e-831a-1465d3896415 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
8312024-11-28T02:00:46.283ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = b7d1f535-f0f7-413e-a51f-a26a201b2a4e roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
8322024-11-28T02:00:46.283ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = b7d1f535-f0f7-413e-a51f-a26a201b2a4e resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
8332024-11-28T02:00:46.285ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2527 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = b7d1f535-f0f7-413e-a51f-a26a201b2a4e response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
8342024-11-28T02:00:46.841ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::b]:12221 remote_addr = [fd00:1122:3344:101::2]:51984
8352024-11-28T02:00:46.841ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:51984 req_id = 4fc5c34e-db4b-49ca-be22-ced84b83241a roles = RoleSet { roles: {} } uri = /metrics/producers
8362024-11-28T02:00:46.842ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:51984 req_id = 4fc5c34e-db4b-49ca-be22-ced84b83241a resource = Database result = Ok(()) uri = /metrics/producers
8372024-11-28T02:00:46.851ZINFOnexus: assigned collector to new producer collector_id = 119eed64-2e24-4d5f-8c42-be6cd8bec8b5 file = nexus/src/app/oximeter.rs:76 producer_id = 27b44fd9-9dc4-42a5-82e8-5084c0434475
8382024-11-28T02:00:46.851ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 10388 local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:51984 req_id = 4fc5c34e-db4b-49ca-be22-ced84b83241a response_code = 201 uri = /metrics/producers
8392024-11-28T02:00:47.305ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = cc7541d0-bb49-45f0-9d3c-83fa086ba930 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
8402024-11-28T02:00:47.306ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = cc7541d0-bb49-45f0-9d3c-83fa086ba930 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
8412024-11-28T02:00:47.308ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2211 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = cc7541d0-bb49-45f0-9d3c-83fa086ba930 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
8422024-11-28T02:00:48.327ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 3c296c5c-5d0a-4142-b5b7-64ba3861f238 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
8432024-11-28T02:00:48.328ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 3c296c5c-5d0a-4142-b5b7-64ba3861f238 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
8442024-11-28T02:00:48.330ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2277 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 3c296c5c-5d0a-4142-b5b7-64ba3861f238 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
8452024-11-28T02:00:49.348ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = f8eec953-58b3-432e-8b1b-1efcf98cd4f6 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
8462024-11-28T02:00:49.349ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = f8eec953-58b3-432e-8b1b-1efcf98cd4f6 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
8472024-11-28T02:00:49.350ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2337 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = f8eec953-58b3-432e-8b1b-1efcf98cd4f6 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
8482024-11-28T02:00:50.369ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = e6ec8141-a133-4079-97a8-a4f3d186724c roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
8492024-11-28T02:00:50.370ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = e6ec8141-a133-4079-97a8-a4f3d186724c resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
8502024-11-28T02:00:50.371ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1839 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = e6ec8141-a133-4079-97a8-a4f3d186724c response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
8512024-11-28T02:00:50.477ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 47 reason = Timeout
8522024-11-28T02:00:50.477ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
8532024-11-28T02:00:50.477ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 47 reason = Timeout
8542024-11-28T02:00:50.477ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
8552024-11-28T02:00:50.478ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
8562024-11-28T02:00:50.478ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
8572024-11-28T02:00:50.479ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8582024-11-28T02:00:50.482ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
8592024-11-28T02:00:50.482ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
8602024-11-28T02:00:50.482ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
8612024-11-28T02:00:50.484ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
8622024-11-28T02:00:50.484ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
8632024-11-28T02:00:50.485ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8642024-11-28T02:00:50.488ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 67ac247f-c934-4c78-857a-fba4fbeb9937, lookup_type: ById(67ac247f-c934-4c78-857a-fba4fbeb9937) } result = Ok(())
8652024-11-28T02:00:50.488ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
8662024-11-28T02:00:50.488ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
8672024-11-28T02:00:50.502ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 67ac247f-c934-4c78-857a-fba4fbeb9937 original_time_created = 2024-11-28 01:53:03.159475 UTC target_id = 67ac247f-c934-4c78-857a-fba4fbeb9937 time_created = 2024-11-28 01:53:03.159475 UTC
8682024-11-28T02:00:50.502ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 24.452584ms iteration = 47 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"67ac247f-c934-4c78-857a-fba4fbeb9937","time_created":"2024-11-28T01:53:03.159475Z"}
8692024-11-28T02:00:50.512ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
8702024-11-28T02:00:50.512ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
8712024-11-28T02:00:50.547ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 69.705361ms iteration = 47 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
8722024-11-28T02:00:51.390ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = d352170e-d8d7-4877-b204-739fc7164558 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
8732024-11-28T02:00:51.391ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = d352170e-d8d7-4877-b204-739fc7164558 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
8742024-11-28T02:00:51.392ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1996 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = d352170e-d8d7-4877-b204-739fc7164558 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
8752024-11-28T02:00:52.414ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 1e2472a2-6cf6-491f-970d-faf1c58fe51e roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
8762024-11-28T02:00:52.414ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 1e2472a2-6cf6-491f-970d-faf1c58fe51e resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
8772024-11-28T02:00:52.416ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2554 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 1e2472a2-6cf6-491f-970d-faf1c58fe51e response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
8782024-11-28T02:00:53.435ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = e8ec6732-246d-41bd-8035-64da945d0b99 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
8792024-11-28T02:00:53.436ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = e8ec6732-246d-41bd-8035-64da945d0b99 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
8802024-11-28T02:00:53.438ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2521 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = e8ec6732-246d-41bd-8035-64da945d0b99 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
8812024-11-28T02:00:54.193ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1260 local_addr = [fd00:1122:3344:101::b]:55621 method = GET remote_addr = [fd00:1122:3344:101::d]:56417 req_id = 784c8c41-53a4-496d-bafd-dfba336676cc response_code = 200 uri = /2469a748-bcd2-4e0d-99cc-f79def0452cc
8822024-11-28T02:00:54.457ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 03656aa3-81c3-4dee-a7eb-828cbe41752c roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
8832024-11-28T02:00:54.457ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 03656aa3-81c3-4dee-a7eb-828cbe41752c resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
8842024-11-28T02:00:54.459ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2035 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 03656aa3-81c3-4dee-a7eb-828cbe41752c response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
8852024-11-28T02:00:55.481ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 15abf98e-95ed-4da8-a172-b73e3d851a7d roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
8862024-11-28T02:00:55.481ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 15abf98e-95ed-4da8-a172-b73e3d851a7d resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
8872024-11-28T02:00:55.482ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1771 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 15abf98e-95ed-4da8-a172-b73e3d851a7d response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
8882024-11-28T02:00:56.501ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 55e169c6-39aa-4e59-9d43-f89451a773f8 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
8892024-11-28T02:00:56.501ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 55e169c6-39aa-4e59-9d43-f89451a773f8 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
8902024-11-28T02:00:56.503ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1837 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 55e169c6-39aa-4e59-9d43-f89451a773f8 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
8912024-11-28T02:00:56.757ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::b]:12221 remote_addr = [fd00:1122:3344:101::2]:42435
8922024-11-28T02:00:56.757ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:42435 req_id = 48f1a80d-213a-4311-b144-d0587db0504f roles = RoleSet { roles: {} } uri = /metrics/producers
8932024-11-28T02:00:56.757ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:42435 req_id = 48f1a80d-213a-4311-b144-d0587db0504f resource = Database result = Ok(()) uri = /metrics/producers
8942024-11-28T02:00:56.768ZINFOnexus: assigned collector to new producer collector_id = 119eed64-2e24-4d5f-8c42-be6cd8bec8b5 file = nexus/src/app/oximeter.rs:76 producer_id = c434b448-5199-466e-af32-4f629e31c7b1
8952024-11-28T02:00:56.768ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 11368 local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:42435 req_id = 48f1a80d-213a-4311-b144-d0587db0504f response_code = 201 uri = /metrics/producers
8962024-11-28T02:00:57.521ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = e60d60f6-e932-4b20-b205-29876ea7bb17 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
8972024-11-28T02:00:57.521ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = e60d60f6-e932-4b20-b205-29876ea7bb17 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
8982024-11-28T02:00:57.523ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2418 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = e60d60f6-e932-4b20-b205-29876ea7bb17 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
8992024-11-28T02:00:58.542ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = b0cff084-b632-4e7f-8c2f-78c93ab574a5 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
9002024-11-28T02:00:58.543ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = b0cff084-b632-4e7f-8c2f-78c93ab574a5 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
9012024-11-28T02:00:58.545ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2550 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = b0cff084-b632-4e7f-8c2f-78c93ab574a5 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
9022024-11-28T02:00:59.565ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = ac6130d0-ff17-4700-8438-ae71beb2ace8 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
9032024-11-28T02:00:59.565ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = ac6130d0-ff17-4700-8438-ae71beb2ace8 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
9042024-11-28T02:00:59.566ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1667 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = ac6130d0-ff17-4700-8438-ae71beb2ace8 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
9052024-11-28T02:01:00.477ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 48 reason = Timeout
9062024-11-28T02:01:00.477ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
9072024-11-28T02:01:00.477ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 48 reason = Timeout
9082024-11-28T02:01:00.477ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9092024-11-28T02:01:00.478ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
9102024-11-28T02:01:00.478ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
9112024-11-28T02:01:00.479ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9122024-11-28T02:01:00.479ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
9132024-11-28T02:01:00.480ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
9142024-11-28T02:01:00.482ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
9152024-11-28T02:01:00.482ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9162024-11-28T02:01:00.482ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
9172024-11-28T02:01:00.484ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9182024-11-28T02:01:00.484ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
9192024-11-28T02:01:00.486ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9202024-11-28T02:01:00.489ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 67ac247f-c934-4c78-857a-fba4fbeb9937, lookup_type: ById(67ac247f-c934-4c78-857a-fba4fbeb9937) } result = Ok(())
9212024-11-28T02:01:00.489ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9222024-11-28T02:01:00.489ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
9232024-11-28T02:01:00.499ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 67ac247f-c934-4c78-857a-fba4fbeb9937 original_time_created = 2024-11-28 01:53:03.159475 UTC target_id = 67ac247f-c934-4c78-857a-fba4fbeb9937 time_created = 2024-11-28 01:53:03.159475 UTC
9242024-11-28T02:01:00.499ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 22.123214ms iteration = 48 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"67ac247f-c934-4c78-857a-fba4fbeb9937","time_created":"2024-11-28T01:53:03.159475Z"}
9252024-11-28T02:01:00.507ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 30.086192ms iteration = 48 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
9262024-11-28T02:01:00.585ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 879ba6d5-114c-4ddd-be41-bf2b6dd1f40a roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
9272024-11-28T02:01:00.586ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 879ba6d5-114c-4ddd-be41-bf2b6dd1f40a resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
9282024-11-28T02:01:00.587ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1900 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 879ba6d5-114c-4ddd-be41-bf2b6dd1f40a response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
9292024-11-28T02:01:01.606ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 45395ce7-9f66-413c-948a-dc1da5e889dd roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
9302024-11-28T02:01:01.607ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 45395ce7-9f66-413c-948a-dc1da5e889dd resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
9312024-11-28T02:01:01.608ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2122 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 45395ce7-9f66-413c-948a-dc1da5e889dd response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
9322024-11-28T02:01:02.628ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 75d54e02-5b14-44f6-9f20-8b63c61339ea roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
9332024-11-28T02:01:02.628ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 75d54e02-5b14-44f6-9f20-8b63c61339ea resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
9342024-11-28T02:01:02.630ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2466 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 75d54e02-5b14-44f6-9f20-8b63c61339ea response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
9352024-11-28T02:01:03.648ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 11043522-fc92-485f-9ec0-b92f3eee5bda roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
9362024-11-28T02:01:03.648ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 11043522-fc92-485f-9ec0-b92f3eee5bda resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
9372024-11-28T02:01:03.650ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2040 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 11043522-fc92-485f-9ec0-b92f3eee5bda response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
9382024-11-28T02:01:04.194ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1397 local_addr = [fd00:1122:3344:101::b]:55621 method = GET remote_addr = [fd00:1122:3344:101::d]:56417 req_id = 80ccab05-35de-4005-a8b6-884f3848613c response_code = 200 uri = /2469a748-bcd2-4e0d-99cc-f79def0452cc
9392024-11-28T02:01:04.668ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = df1197aa-e2a1-41cc-96fa-ff480d38e339 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
9402024-11-28T02:01:04.668ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = df1197aa-e2a1-41cc-96fa-ff480d38e339 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
9412024-11-28T02:01:04.670ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2031 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = df1197aa-e2a1-41cc-96fa-ff480d38e339 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
9422024-11-28T02:01:05.690ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = a98e7015-fba1-43a0-ba7f-a8f8fb6b3db5 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
9432024-11-28T02:01:05.691ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = a98e7015-fba1-43a0-ba7f-a8f8fb6b3db5 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
9442024-11-28T02:01:05.692ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1873 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = a98e7015-fba1-43a0-ba7f-a8f8fb6b3db5 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
9452024-11-28T02:01:06.710ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 615ff28a-a7ba-4512-ab2c-ccc256f0366a roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
9462024-11-28T02:01:06.710ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 615ff28a-a7ba-4512-ab2c-ccc256f0366a resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
9472024-11-28T02:01:06.712ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2321 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 615ff28a-a7ba-4512-ab2c-ccc256f0366a response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
9482024-11-28T02:01:07.732ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 8e73c309-ada5-40d9-b917-135e1b4b6bdb roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
9492024-11-28T02:01:07.732ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 8e73c309-ada5-40d9-b917-135e1b4b6bdb resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
9502024-11-28T02:01:07.734ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2458 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 8e73c309-ada5-40d9-b917-135e1b4b6bdb response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
9512024-11-28T02:01:08.752ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = f57e0d16-cd6e-4c8c-82b1-784c95b150d5 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
9522024-11-28T02:01:08.752ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = f57e0d16-cd6e-4c8c-82b1-784c95b150d5 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
9532024-11-28T02:01:08.753ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1454 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = f57e0d16-cd6e-4c8c-82b1-784c95b150d5 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
9542024-11-28T02:01:09.774ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 96cead2f-4152-4947-973c-a7aa29101a3b roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
9552024-11-28T02:01:09.774ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 96cead2f-4152-4947-973c-a7aa29101a3b resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
9562024-11-28T02:01:09.776ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2560 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 96cead2f-4152-4947-973c-a7aa29101a3b response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
9572024-11-28T02:01:10.477ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 11 reason = Timeout
9582024-11-28T02:01:10.477ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 17 reason = Timeout
9592024-11-28T02:01:10.477ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
9602024-11-28T02:01:10.477ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 9 reason = Timeout
9612024-11-28T02:01:10.477ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 17 reason = Timeout
9622024-11-28T02:01:10.477ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
9632024-11-28T02:01:10.477ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 17 reason = Timeout
9642024-11-28T02:01:10.477ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
9652024-11-28T02:01:10.477ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
9662024-11-28T02:01:10.477ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 17 reason = Timeout
9672024-11-28T02:01:10.477ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
9682024-11-28T02:01:10.477ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 49 reason = Timeout
9692024-11-28T02:01:10.477ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
9702024-11-28T02:01:10.478ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 10 reason = Timeout
9712024-11-28T02:01:10.478ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
9722024-11-28T02:01:10.478ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9732024-11-28T02:01:10.478ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
9742024-11-28T02:01:10.478ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 17 reason = Timeout
9752024-11-28T02:01:10.478ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 17 reason = Timeout
9762024-11-28T02:01:10.478ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
9772024-11-28T02:01:10.478ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
9782024-11-28T02:01:10.478ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 20 reason = Timeout
9792024-11-28T02:01:10.478ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 10 reason = Timeout
9802024-11-28T02:01:10.478ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
9812024-11-28T02:01:10.478ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
9822024-11-28T02:01:10.478ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 11 reason = Timeout
9832024-11-28T02:01:10.478ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 9 reason = Timeout
9842024-11-28T02:01:10.478ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
9852024-11-28T02:01:10.478ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 17 reason = Timeout
9862024-11-28T02:01:10.478ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
9872024-11-28T02:01:10.478ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 17 reason = Timeout
9882024-11-28T02:01:10.478ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
9892024-11-28T02:01:10.478ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 10 reason = Timeout
9902024-11-28T02:01:10.478ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 10 reason = Timeout
9912024-11-28T02:01:10.478ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:83 reason = blueprint disabled target_id = 67ac247f-c934-4c78-857a-fba4fbeb9937
9922024-11-28T02:01:10.478ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 15.791\u{b5}s iteration = 10 status = {"enabled":false,"target_id":"67ac247f-c934-4c78-857a-fba4fbeb9937"}
9932024-11-28T02:01:10.478ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 17 reason = Timeout
9942024-11-28T02:01:10.478ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 17 reason = Timeout
9952024-11-28T02:01:10.478ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
9962024-11-28T02:01:10.478ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 9 reason = Timeout
9972024-11-28T02:01:10.478ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:56
9982024-11-28T02:01:10.478ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
9992024-11-28T02:01:10.478ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 18 reason = Timeout
10002024-11-28T02:01:10.478ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
10012024-11-28T02:01:10.478ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 9 reason = Timeout
10022024-11-28T02:01:10.478ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
10032024-11-28T02:01:10.478ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
10042024-11-28T02:01:10.478ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 17 reason = Timeout
10052024-11-28T02:01:10.478ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
10062024-11-28T02:01:10.478ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 10 reason = Timeout
10072024-11-28T02:01:10.478ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 17 reason = Timeout
10082024-11-28T02:01:10.478ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
10092024-11-28T02:01:10.478ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 10 reason = Timeout
10102024-11-28T02:01:10.478ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
10112024-11-28T02:01:10.478ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
10122024-11-28T02:01:10.478ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 9 reason = Timeout
10132024-11-28T02:01:10.478ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2024-11-28 01:51:10.478313071 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
10142024-11-28T02:01:10.478ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 49 reason = Timeout
10152024-11-28T02:01:10.478ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
10162024-11-28T02:01:10.478ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
10172024-11-28T02:01:10.478ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 17 reason = Timeout
10182024-11-28T02:01:10.478ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
10192024-11-28T02:01:10.478ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
10202024-11-28T02:01:10.478ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
10212024-11-28T02:01:10.478ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
10222024-11-28T02:01:10.478ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
10232024-11-28T02:01:10.478ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
10242024-11-28T02:01:10.478ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
10252024-11-28T02:01:10.478ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
10262024-11-28T02:01:10.478ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
10272024-11-28T02:01:10.478ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
10282024-11-28T02:01:10.478ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
10292024-11-28T02:01:10.478ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
10302024-11-28T02:01:10.478ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
10312024-11-28T02:01:10.478ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
10322024-11-28T02:01:10.478ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
10332024-11-28T02:01:10.478ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
10342024-11-28T02:01:10.478ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
10352024-11-28T02:01:10.478ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
10362024-11-28T02:01:10.478ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
10372024-11-28T02:01:10.478ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
10382024-11-28T02:01:10.478ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
10392024-11-28T02:01:10.478ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
10402024-11-28T02:01:10.478ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
10412024-11-28T02:01:10.478ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
10422024-11-28T02:01:10.478ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("36d4f615-7461-4364-960c-89ece3aaabaf.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("eb274350-c0b0-4bce-9c8a-502880f2c905.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("eb274350-c0b0-4bce-9c8a-502880f2c905.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::8))) }], valid_until: Instant { tv_sec: 1336, tv_nsec: 949718681 } })
10432024-11-28T02:01:10.478ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
10442024-11-28T02:01:10.479ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("41ea4a99-3038-474e-bf28-2ad338f3243f.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("9402a51b-d30f-4544-93ec-0f5dc7a485e4.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("d23bc5f2-6b92-4992-bdd2-8695e5bf9c0f.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("d23bc5f2-6b92-4992-bdd2-8695e5bf9c0f.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:1::1))) }], valid_until: Instant { tv_sec: 1336, tv_nsec: 949775742 } })
10452024-11-28T02:01:10.479ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } dns_group = external
10462024-11-28T02:01:10.479ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.230621ms iteration = 10 status = {"addresses":["[fd00:1122:3344:101::9]:5353","[fd00:1122:3344:101::8]:5353"]}
10472024-11-28T02:01:10.479ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } dns_group = internal
10482024-11-28T02:01:10.479ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.308783ms iteration = 10 status = {"addresses":["[fd00:1122:3344:3::1]:5353","[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:1::1]:5353"]}
10492024-11-28T02:01:10.482ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10502024-11-28T02:01:10.482ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
10512024-11-28T02:01:10.483ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
10522024-11-28T02:01:10.485ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 7.369528ms iteration = 17 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
10532024-11-28T02:01:10.486ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
10542024-11-28T02:01:10.486ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
10552024-11-28T02:01:10.486ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
10562024-11-28T02:01:10.488ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-313847b3-d670-4167-9b09-9e7d3dfcb2a4.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-313847b3-d670-4167-9b09-9e7d3dfcb2a4.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1336, tv_nsec: 959286177 } })
10572024-11-28T02:01:10.488ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1103
10582024-11-28T02:01:10.489ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10592024-11-28T02:01:10.492ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 14.814367ms iteration = 9 status = {"errors":[],"found_port_ok":[]}
10602024-11-28T02:01:10.493ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
10612024-11-28T02:01:10.493ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
10622024-11-28T02:01:10.493ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
10632024-11-28T02:01:10.503ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10642024-11-28T02:01:10.507ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
10652024-11-28T02:01:10.507ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
10662024-11-28T02:01:10.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
10672024-11-28T02:01:10.507ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
10682024-11-28T02:01:10.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
10692024-11-28T02:01:10.511ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-313847b3-d670-4167-9b09-9e7d3dfcb2a4.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-313847b3-d670-4167-9b09-9e7d3dfcb2a4.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1336, tv_nsec: 981780928 } })
10702024-11-28T02:01:10.511ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1103
10712024-11-28T02:01:10.512ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2024-11-28 01:51:10.478313071 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
10722024-11-28T02:01:10.512ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 34.299206ms iteration = 9 status = {"expiration":"2024-11-28T01:51:10.478313071Z","pruned":[]}
10732024-11-28T02:01:10.515ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-313847b3-d670-4167-9b09-9e7d3dfcb2a4.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-313847b3-d670-4167-9b09-9e7d3dfcb2a4.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1336, tv_nsec: 985752367 } })
10742024-11-28T02:01:10.515ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1103 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc
10752024-11-28T02:01:10.547ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
10762024-11-28T02:01:10.547ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
10772024-11-28T02:01:10.547ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
10782024-11-28T02:01:10.547ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
10792024-11-28T02:01:10.550ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
10802024-11-28T02:01:10.550ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
10812024-11-28T02:01:10.550ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 72.122732ms iteration = 17 status = {"errors":[],"records_set_to_done":[]}
10822024-11-28T02:01:10.551ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10832024-11-28T02:01:10.552ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10842024-11-28T02:01:10.553ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
10852024-11-28T02:01:10.553ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10862024-11-28T02:01:10.553ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
10872024-11-28T02:01:10.554ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
10882024-11-28T02:01:10.554ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
10892024-11-28T02:01:10.554ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
10902024-11-28T02:01:10.555ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10912024-11-28T02:01:10.556ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10922024-11-28T02:01:10.557ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 79.323678ms iteration = 17 status = {"errors":[],"garbage_collect_requested":[]}
10932024-11-28T02:01:10.557ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
10942024-11-28T02:01:10.558ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
10952024-11-28T02:01:10.558ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
10962024-11-28T02:01:10.558ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
10972024-11-28T02:01:10.558ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
10982024-11-28T02:01:10.559ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
10992024-11-28T02:01:10.559ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
11002024-11-28T02:01:10.559ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:171 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
11012024-11-28T02:01:10.559ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 81.213991ms iteration = 9 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
11022024-11-28T02:01:10.559ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
11032024-11-28T02:01:10.559ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
11042024-11-28T02:01:10.559ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
11052024-11-28T02:01:10.560ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
11062024-11-28T02:01:10.560ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
11072024-11-28T02:01:10.560ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
11082024-11-28T02:01:10.561ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11092024-11-28T02:01:10.563ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
11102024-11-28T02:01:10.563ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
11112024-11-28T02:01:10.564ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
11122024-11-28T02:01:10.564ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
11132024-11-28T02:01:10.564ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
11142024-11-28T02:01:10.565ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
11152024-11-28T02:01:10.565ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
11162024-11-28T02:01:10.566ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
11172024-11-28T02:01:10.566ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
11182024-11-28T02:01:10.567ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11192024-11-28T02:01:10.567ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11202024-11-28T02:01:10.568ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11212024-11-28T02:01:10.569ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 91.237374ms iteration = 17 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
11222024-11-28T02:01:10.570ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
11232024-11-28T02:01:10.570ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
11242024-11-28T02:01:10.570ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
11252024-11-28T02:01:10.572ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
11262024-11-28T02:01:10.572ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
11272024-11-28T02:01:10.572ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
11282024-11-28T02:01:10.572ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
11292024-11-28T02:01:10.572ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
11302024-11-28T02:01:10.572ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
11312024-11-28T02:01:10.573ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11322024-11-28T02:01:10.574ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
11332024-11-28T02:01:10.575ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
11342024-11-28T02:01:10.577ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
11352024-11-28T02:01:10.577ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:140 found = 0 not_ready_to_be_deleted = 0
11362024-11-28T02:01:10.577ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 99.00822ms iteration = 9 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
11372024-11-28T02:01:10.577ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 67ac247f-c934-4c78-857a-fba4fbeb9937, lookup_type: ById(67ac247f-c934-4c78-857a-fba4fbeb9937) } result = Ok(())
11382024-11-28T02:01:10.577ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
11392024-11-28T02:01:10.578ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
11402024-11-28T02:01:10.580ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
11412024-11-28T02:01:10.580ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 102.764845ms iteration = 17 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
11422024-11-28T02:01:10.593ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 67ac247f-c934-4c78-857a-fba4fbeb9937 original_time_created = 2024-11-28 01:53:03.159475 UTC target_id = 67ac247f-c934-4c78-857a-fba4fbeb9937 time_created = 2024-11-28 01:53:03.159475 UTC
11432024-11-28T02:01:10.593ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 115.854662ms iteration = 49 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"67ac247f-c934-4c78-857a-fba4fbeb9937","time_created":"2024-11-28T01:53:03.159475Z"}
11442024-11-28T02:01:10.598ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
11452024-11-28T02:01:10.598ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
11462024-11-28T02:01:10.598ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
11472024-11-28T02:01:10.598ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
11482024-11-28T02:01:10.598ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
11492024-11-28T02:01:10.600ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11502024-11-28T02:01:10.601ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
11512024-11-28T02:01:10.601ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
11522024-11-28T02:01:10.603ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
11532024-11-28T02:01:10.604ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
11542024-11-28T02:01:10.604ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
11552024-11-28T02:01:10.605ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 127.145539ms iteration = 49 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
11562024-11-28T02:01:10.605ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
11572024-11-28T02:01:10.605ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
11582024-11-28T02:01:10.607ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
11592024-11-28T02:01:10.607ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
11602024-11-28T02:01:10.607ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
11612024-11-28T02:01:10.608ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
11622024-11-28T02:01:10.609ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11632024-11-28T02:01:10.610ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
11642024-11-28T02:01:10.610ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
11652024-11-28T02:01:10.610ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
11662024-11-28T02:01:10.610ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
11672024-11-28T02:01:10.610ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
11682024-11-28T02:01:10.612ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11692024-11-28T02:01:10.613ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11702024-11-28T02:01:10.614ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
11712024-11-28T02:01:10.614ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
11722024-11-28T02:01:10.614ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
11732024-11-28T02:01:10.616ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 313847b3-d670-4167-9b09-9e7d3dfcb2a4, lookup_type: ById(313847b3-d670-4167-9b09-9e7d3dfcb2a4) } result = Ok(())
11742024-11-28T02:01:10.616ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
11752024-11-28T02:01:10.616ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
11762024-11-28T02:01:10.617ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
11772024-11-28T02:01:10.617ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
11782024-11-28T02:01:10.618ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
11792024-11-28T02:01:10.619ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
11802024-11-28T02:01:10.619ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
11812024-11-28T02:01:10.619ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
11822024-11-28T02:01:10.620ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
11832024-11-28T02:01:10.624ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 146.383503ms iteration = 17 status = 0
11842024-11-28T02:01:10.624ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 146.744439ms iteration = 17 status = {"errors":[],"requests_created_ok":[],"start_invoked_ok":[]}
11852024-11-28T02:01:10.643ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
11862024-11-28T02:01:10.646ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
11872024-11-28T02:01:10.650ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
11882024-11-28T02:01:10.650ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = a7e6af61-408c-483d-b043-c27ccf55708e dns_zone_name = oxide.test found_so_far = 0 version = 2
11892024-11-28T02:01:10.650ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
11902024-11-28T02:01:10.650ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
11912024-11-28T02:01:10.651ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 173.783289ms iteration = 17 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[]}
11922024-11-28T02:01:10.660ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
11932024-11-28T02:01:10.661ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
11942024-11-28T02:01:10.661ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
11952024-11-28T02:01:10.661ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
11962024-11-28T02:01:10.671ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
11972024-11-28T02:01:10.671ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 808a324e-c913-4bfc-91a5-d43c961cd511 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
11982024-11-28T02:01:10.671ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
11992024-11-28T02:01:10.671ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
12002024-11-28T02:01:10.674ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
12012024-11-28T02:01:10.674ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
12022024-11-28T02:01:10.674ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 196.408301ms iteration = 17 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
12032024-11-28T02:01:10.685ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
12042024-11-28T02:01:10.685ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
12052024-11-28T02:01:10.686ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "d51f2e02-ff07-4977-8a67-726705c29e96", "content-length": "26", "date": "Thu, 28 Nov 2024 02:01:10 GMT"} })
12062024-11-28T02:01:10.686ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
12072024-11-28T02:01:10.688ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
12082024-11-28T02:01:10.722ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 244.402525ms iteration = 20 status = {"physical_disks_added":0}
12092024-11-28T02:01:10.732ZWARNnexus (BackgroundTasks): instance reincarnation completed with errors background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:103 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0 query_errors = 0 restart_errors = 0
12102024-11-28T02:01:10.733ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 255.096111ms iteration = 9 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
12112024-11-28T02:01:10.753ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
12122024-11-28T02:01:10.753ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
12132024-11-28T02:01:10.753ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "6671cc28-1c72-4108-9192-e91497d84f3a", "content-length": "26", "date": "Thu, 28 Nov 2024 02:01:10 GMT"} })
12142024-11-28T02:01:10.753ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
12152024-11-28T02:01:10.755ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
12162024-11-28T02:01:10.795ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 405e2395-c462-476a-aa32-6bf0d663d84e roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
12172024-11-28T02:01:10.795ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 405e2395-c462-476a-aa32-6bf0d663d84e resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
12182024-11-28T02:01:10.803ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1111 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc zone_address = fd00:1122:3344:101::2
12192024-11-28T02:01:10.803ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
12202024-11-28T02:01:10.803ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "8bcc2387-df14-4d0f-b9e3-72944db3167a", "content-length": "26", "date": "Thu, 28 Nov 2024 02:01:10 GMT"} })
12212024-11-28T02:01:10.803ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1116 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
12222024-11-28T02:01:10.806ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc
12232024-11-28T02:01:10.820ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
12242024-11-28T02:01:10.827ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
12252024-11-28T02:01:10.828ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "13e166a1-d71e-4089-9dc0-0234acabe0ec", "date": "Thu, 28 Nov 2024 02:01:10 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
12262024-11-28T02:01:10.828ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
12272024-11-28T02:01:10.830ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "ecbd53ed-ce09-42eb-95b8-63303370a7e2", "date": "Thu, 28 Nov 2024 02:01:10 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
12282024-11-28T02:01:10.830ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
12292024-11-28T02:01:10.832ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
12302024-11-28T02:01:10.832ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 354.192292ms iteration = 11 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
12312024-11-28T02:01:10.834ZINFOnexus (BackgroundTasks): VPC route manager sled 313847b3-d670-4167-9b09-9e7d3dfcb2a4 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
12322024-11-28T02:01:10.834ZDEBGnexus (BackgroundTasks): client request SledAgent = 313847b3-d670-4167-9b09-9e7d3dfcb2a4 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
12332024-11-28T02:01:10.834ZDEBGnexus (BackgroundTasks): client response SledAgent = 313847b3-d670-4167-9b09-9e7d3dfcb2a4 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "a0d0baf4-5a21-4ede-b2eb-e290d488e497", "content-length": "372", "date": "Thu, 28 Nov 2024 02:01:10 GMT"} })
12342024-11-28T02:01:10.834ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
12352024-11-28T02:01:10.835ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
12362024-11-28T02:01:10.836ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12372024-11-28T02:01:10.837ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12382024-11-28T02:01:10.837ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12392024-11-28T02:01:10.838ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 42675 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 405e2395-c462-476a-aa32-6bf0d663d84e response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
12402024-11-28T02:01:10.840ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
12412024-11-28T02:01:10.840ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
12422024-11-28T02:01:10.840ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
12432024-11-28T02:01:10.840ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
12442024-11-28T02:01:10.840ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
12452024-11-28T02:01:10.840ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
12462024-11-28T02:01:10.842ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
12472024-11-28T02:01:10.842ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
12482024-11-28T02:01:10.842ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
12492024-11-28T02:01:10.842ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = a7e6af61-408c-483d-b043-c27ccf55708e dns_zone_name = oxide.test found_so_far = 1 version = 2
12502024-11-28T02:01:10.842ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
12512024-11-28T02:01:10.842ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2024-11-28 01:53:43.351579 UTC dns_group = external generation = 2
12522024-11-28T02:01:10.842ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 364.530601ms iteration = 10 status = {"generation":2}
12532024-11-28T02:01:10.843ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
12542024-11-28T02:01:10.843ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
12552024-11-28T02:01:10.844ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 808a324e-c913-4bfc-91a5-d43c961cd511 dns_zone_name = control-plane.oxide.internal found_so_far = 51 version = 1
12562024-11-28T02:01:10.844ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
12572024-11-28T02:01:10.844ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2024-11-28 01:53:42.502562 UTC dns_group = internal generation = 1
12582024-11-28T02:01:10.844ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 366.408284ms iteration = 10 status = {"generation":1}
12592024-11-28T02:01:10.845ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12602024-11-28T02:01:10.848ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
12612024-11-28T02:01:10.848ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
12622024-11-28T02:01:10.848ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
12632024-11-28T02:01:10.849ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12642024-11-28T02:01:10.852ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
12652024-11-28T02:01:10.852ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
12662024-11-28T02:01:10.852ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
12672024-11-28T02:01:10.877ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2024-11-28T01:53:08.890632Z, time_modified: 2024-11-28T01:53:08.890632Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 7e159834-997a-4c03-b9db-1e91d783221c, db_silo: Silo { identity: SiloIdentity { id: 7e159834-997a-4c03-b9db-1e91d783221c, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-11-28T01:53:43.308696Z, time_modified: 2024-11-28T01:53:43.308696Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "bc39d1b3b4bb84a49d6458ed7aedc5bca9e77742c7b35882b64d188c0602d306" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 7e159834-997a-4c03-b9db-1e91d783221c, db_silo: Silo { identity: SiloIdentity { id: 7e159834-997a-4c03-b9db-1e91d783221c, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-11-28T01:53:43.308696Z, time_modified: 2024-11-28T01:53:43.308696Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "bc39d1b3b4bb84a49d6458ed7aedc5bca9e77742c7b35882b64d188c0602d306" }] }) }
12682024-11-28T02:01:10.877ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 399.336797ms iteration = 10 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"7e159834-997a-4c03-b9db-1e91d783221c","tls_certs":["bc39d1b3b4bb84a49d6458ed7aedc5bca9e77742c7b35882b64d188c0602d306"]}},"default_endpoint":{"silo_id":"7e159834-997a-4c03-b9db-1e91d783221c","tls_certs":["bc39d1b3b4bb84a49d6458ed7aedc5bca9e77742c7b35882b64d188c0602d306"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
12692024-11-28T02:01:10.891ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
12702024-11-28T02:01:10.891ZDEBGnexus (BackgroundTasks): client request SledAgent = 313847b3-d670-4167-9b09-9e7d3dfcb2a4 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
12712024-11-28T02:01:10.891ZDEBGnexus (BackgroundTasks): client response SledAgent = 313847b3-d670-4167-9b09-9e7d3dfcb2a4 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "52576a85-4512-4ce4-8cfb-f2832611e2cd", "date": "Thu, 28 Nov 2024 02:01:10 GMT"} })
12722024-11-28T02:01:10.891ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
12732024-11-28T02:01:10.892ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
12742024-11-28T02:01:10.895ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
12752024-11-28T02:01:10.895ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
12762024-11-28T02:01:10.907ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
12772024-11-28T02:01:10.907ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
12782024-11-28T02:01:10.911ZINFOnexus (BackgroundTasks): VPC route manager sled 313847b3-d670-4167-9b09-9e7d3dfcb2a4 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
12792024-11-28T02:01:10.911ZDEBGnexus (BackgroundTasks): client request SledAgent = 313847b3-d670-4167-9b09-9e7d3dfcb2a4 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
12802024-11-28T02:01:10.911ZDEBGnexus (BackgroundTasks): client response SledAgent = 313847b3-d670-4167-9b09-9e7d3dfcb2a4 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "84aa8aad-f63d-4b6a-a465-4072c19761f5", "date": "Thu, 28 Nov 2024 02:01:10 GMT"} })
12812024-11-28T02:01:10.913ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 435.712688ms iteration = 18 status = {}
12822024-11-28T02:01:10.945ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 466.837138ms iteration = 17 status = {}
12832024-11-28T02:01:10.968ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
12842024-11-28T02:01:10.968ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "e0a7db43-8c6e-4072-84fc-dfc12ad4d54a", "date": "Thu, 28 Nov 2024 02:01:10 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
12852024-11-28T02:01:10.968ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
12862024-11-28T02:01:10.970ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "93359ef8-ec4b-432e-8793-44eff4b63437", "date": "Thu, 28 Nov 2024 02:01:10 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
12872024-11-28T02:01:10.970ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
12882024-11-28T02:01:10.972ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "be45eebe-1435-4efb-8ddd-f923e8b61ed1", "date": "Thu, 28 Nov 2024 02:01:10 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
12892024-11-28T02:01:10.972ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
12902024-11-28T02:01:10.975ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
12912024-11-28T02:01:10.975ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 497.411859ms iteration = 11 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
12922024-11-28T02:01:10.975ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
12932024-11-28T02:01:10.975ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "dd0bfbfb-871a-4f07-9f8d-fedf3951d213", "content-length": "1", "date": "Thu, 28 Nov 2024 02:01:10 GMT"} })
12942024-11-28T02:01:10.978ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
12952024-11-28T02:01:10.978ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
12962024-11-28T02:01:10.980ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 502.635691ms iteration = 17 status = null
12972024-11-28T02:01:10.985ZDEBGnexus (BackgroundTasks): client request SledAgent = 313847b3-d670-4167-9b09-9e7d3dfcb2a4 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
12982024-11-28T02:01:10.986ZDEBGnexus (BackgroundTasks): client response SledAgent = 313847b3-d670-4167-9b09-9e7d3dfcb2a4 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "20e8fbcf-8697-40be-8167-0f1415811190", "content-length": "691", "date": "Thu, 28 Nov 2024 02:01:10 GMT"} })
12992024-11-28T02:01:10.986ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
13002024-11-28T02:01:10.986ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
13012024-11-28T02:01:10.986ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
13022024-11-28T02:01:10.988ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 510.199792ms iteration = 17 status = {}
13032024-11-28T02:01:11.033ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
13042024-11-28T02:01:11.034ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
13052024-11-28T02:01:11.036ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 4bb2986f-2e92-4ac3-9dc1-591b6bb60b3e, rack_id: 92a8ff6e-21d3-4c28-bcad-e323847985dc, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(ea300318-9d62-42ed-80ea-21c86590a166) } rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc switch_location = Switch0
13062024-11-28T02:01:11.036ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
13072024-11-28T02:01:11.036ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
13082024-11-28T02:01:11.046ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
13092024-11-28T02:01:11.046ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "0bd4f36c-ad0f-4f9c-9661-f68551892d7e", "content-length": "134", "date": "Thu, 28 Nov 2024 02:01:10 GMT"} })
13102024-11-28T02:01:11.046ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1746 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
13112024-11-28T02:01:11.046ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1773 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = ea300318-9d62-42ed-80ea-21c86590a166
13122024-11-28T02:01:11.047ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
13132024-11-28T02:01:11.047ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
13142024-11-28T02:01:11.047ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
13152024-11-28T02:01:11.047ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
13162024-11-28T02:01:11.047ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
13172024-11-28T02:01:11.047ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1920 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
13182024-11-28T02:01:11.047ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
13192024-11-28T02:01:11.047ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
13202024-11-28T02:01:11.047ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc
13212024-11-28T02:01:11.047ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
13222024-11-28T02:01:11.047ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
13232024-11-28T02:01:11.050ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
13242024-11-28T02:01:11.050ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "ce487105-a28e-48ed-ad4b-b752e7537fbc", "content-length": "2", "date": "Thu, 28 Nov 2024 02:01:10 GMT"} })
13252024-11-28T02:01:11.050ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
13262024-11-28T02:01:11.050ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "44a25d73-5cce-403e-af52-3ba8ba93a03a", "content-length": "52", "date": "Thu, 28 Nov 2024 02:01:10 GMT"} })
13272024-11-28T02:01:11.050ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc switch_location = Switch0
13282024-11-28T02:01:11.050ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
13292024-11-28T02:01:11.194ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "ba6ac609-d32a-41b4-a50d-b13ee3a08664", "date": "Thu, 28 Nov 2024 02:01:10 GMT"} })
13302024-11-28T02:01:11.194ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc switch = Switch0
13312024-11-28T02:01:11.194ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
13322024-11-28T02:01:11.194ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "9473d11c-7fde-4a24-8262-244b1d8d26c6", "content-length": "682", "date": "Thu, 28 Nov 2024 02:01:11 GMT"} })
13332024-11-28T02:01:11.194ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
13342024-11-28T02:01:11.194ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
13352024-11-28T02:01:11.198ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
13362024-11-28T02:01:11.199ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
13372024-11-28T02:01:11.202ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
13382024-11-28T02:01:11.202ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
13392024-11-28T02:01:11.204ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
13402024-11-28T02:01:11.205ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
13412024-11-28T02:01:11.212ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 734.284114ms iteration = 17 status = {}
13422024-11-28T02:01:11.212ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:40402 req_id = 189263cd-371a-4cbd-aaa7-52cb1873da92 roles = RoleSet { roles: {} } uri = /nat/ipv4/changeset/6?limit=100
13432024-11-28T02:01:11.212ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:40402 req_id = 189263cd-371a-4cbd-aaa7-52cb1873da92 resource = Database result = Ok(()) uri = /nat/ipv4/changeset/6?limit=100
13442024-11-28T02:01:11.215ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2895 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:40402 req_id = 189263cd-371a-4cbd-aaa7-52cb1873da92 response_code = 200 uri = /nat/ipv4/changeset/6?limit=100
13452024-11-28T02:01:11.858ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 9e88ccba-3d39-41c6-bb6b-55f6e5404eec roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
13462024-11-28T02:01:11.858ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 9e88ccba-3d39-41c6-bb6b-55f6e5404eec resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
13472024-11-28T02:01:11.860ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2152 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 9e88ccba-3d39-41c6-bb6b-55f6e5404eec response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
13482024-11-28T02:01:12.878ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 8a3ce95d-d3bd-4120-a59a-19d4f65c6a3d roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
13492024-11-28T02:01:12.879ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 8a3ce95d-d3bd-4120-a59a-19d4f65c6a3d resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
13502024-11-28T02:01:12.880ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1952 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 8a3ce95d-d3bd-4120-a59a-19d4f65c6a3d response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
13512024-11-28T02:01:13.899ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = ae747a18-6945-4001-891c-5025bf3d3131 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
13522024-11-28T02:01:13.900ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = ae747a18-6945-4001-891c-5025bf3d3131 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
13532024-11-28T02:01:13.901ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2182 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = ae747a18-6945-4001-891c-5025bf3d3131 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
13542024-11-28T02:01:14.194ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1551 local_addr = [fd00:1122:3344:101::b]:55621 method = GET remote_addr = [fd00:1122:3344:101::d]:56417 req_id = 520aa656-501e-41b3-9362-030f85c10c88 response_code = 200 uri = /2469a748-bcd2-4e0d-99cc-f79def0452cc
13552024-11-28T02:01:14.920ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 95c7db09-1402-45c0-b3a0-36fbda3ced6b roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
13562024-11-28T02:01:14.920ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 95c7db09-1402-45c0-b3a0-36fbda3ced6b resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
13572024-11-28T02:01:14.922ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1548 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 95c7db09-1402-45c0-b3a0-36fbda3ced6b response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
13582024-11-28T02:01:15.212ZDEBGnexus (producer-registration-task): registering / renewing oximeter producer lease with Nexus
13592024-11-28T02:01:15.212ZDEBGnexus (producer-registration-task): using nexus address for registration addr = [fd00:1122:3344:101::b]:12221
13602024-11-28T02:01:15.242ZDEBGnexus (producer-registration-task): client request body = Some(Body) method = POST uri = http://[fd00:1122:3344:101::b]:12221/metrics/producers
13612024-11-28T02:01:15.242ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::b]:12221 remote_addr = [fd00:1122:3344:101::b]:42519
13622024-11-28T02:01:15.242ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::b]:42519 req_id = 82e4ba92-f892-4891-a1f7-69b6350482f8 roles = RoleSet { roles: {} } uri = /metrics/producers
13632024-11-28T02:01:15.243ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::b]:42519 req_id = 82e4ba92-f892-4891-a1f7-69b6350482f8 resource = Database result = Ok(()) uri = /metrics/producers
13642024-11-28T02:01:15.277ZINFOnexus: assigned collector to new producer collector_id = 119eed64-2e24-4d5f-8c42-be6cd8bec8b5 file = nexus/src/app/oximeter.rs:76 producer_id = 2469a748-bcd2-4e0d-99cc-f79def0452cc
13652024-11-28T02:01:15.277ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 35004 local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::b]:42519 req_id = 82e4ba92-f892-4891-a1f7-69b6350482f8 response_code = 201 uri = /metrics/producers
13662024-11-28T02:01:15.277ZDEBGnexus (producer-registration-task): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::b]:12221/metrics/producers", status: 201, headers: {"content-type": "application/json", "x-request-id": "82e4ba92-f892-4891-a1f7-69b6350482f8", "content-length": "41", "date": "Thu, 28 Nov 2024 02:01:15 GMT"} })
13672024-11-28T02:01:15.279ZDEBGnexus (producer-registration-task): registered with nexus successfully
13682024-11-28T02:01:15.279ZDEBGnexus (producer-registration-task): pausing until time to renew lease lease_duration = 600s wait_period = 150s
13692024-11-28T02:01:15.941ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = e73b9459-d752-480e-96fd-83841ae2456c roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
13702024-11-28T02:01:15.942ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = e73b9459-d752-480e-96fd-83841ae2456c resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
13712024-11-28T02:01:15.943ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2096 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = e73b9459-d752-480e-96fd-83841ae2456c response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
13722024-11-28T02:01:16.963ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 4a3e632e-e27e-4301-af98-a6fdad5852bb roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
13732024-11-28T02:01:16.963ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 4a3e632e-e27e-4301-af98-a6fdad5852bb resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
13742024-11-28T02:01:16.965ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1918 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 4a3e632e-e27e-4301-af98-a6fdad5852bb response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
13752024-11-28T02:01:17.989ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 7669277c-4ebf-486c-b41f-f492b5f057ac roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
13762024-11-28T02:01:17.989ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 7669277c-4ebf-486c-b41f-f492b5f057ac resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
13772024-11-28T02:01:17.991ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2315 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 7669277c-4ebf-486c-b41f-f492b5f057ac response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
13782024-11-28T02:01:19.010ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = a87bcb3d-9949-4b43-9e5f-1c2a814af75b roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
13792024-11-28T02:01:19.011ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = a87bcb3d-9949-4b43-9e5f-1c2a814af75b resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
13802024-11-28T02:01:19.012ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2071 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = a87bcb3d-9949-4b43-9e5f-1c2a814af75b response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
13812024-11-28T02:01:20.032ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 6d9033cc-213f-4eea-bb73-c5e2d81eb093 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
13822024-11-28T02:01:20.032ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 6d9033cc-213f-4eea-bb73-c5e2d81eb093 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
13832024-11-28T02:01:20.034ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2195 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 6d9033cc-213f-4eea-bb73-c5e2d81eb093 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
13842024-11-28T02:01:20.478ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 50 reason = Timeout
13852024-11-28T02:01:20.478ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
13862024-11-28T02:01:20.478ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 50 reason = Timeout
13872024-11-28T02:01:20.478ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
13882024-11-28T02:01:20.478ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
13892024-11-28T02:01:20.478ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
13902024-11-28T02:01:20.480ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
13912024-11-28T02:01:20.480ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
13922024-11-28T02:01:20.481ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13932024-11-28T02:01:20.484ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
13942024-11-28T02:01:20.484ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
13952024-11-28T02:01:20.484ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
13962024-11-28T02:01:20.486ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
13972024-11-28T02:01:20.487ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
13982024-11-28T02:01:20.488ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13992024-11-28T02:01:20.491ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 67ac247f-c934-4c78-857a-fba4fbeb9937, lookup_type: ById(67ac247f-c934-4c78-857a-fba4fbeb9937) } result = Ok(())
14002024-11-28T02:01:20.491ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
14012024-11-28T02:01:20.491ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
14022024-11-28T02:01:20.502ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 67ac247f-c934-4c78-857a-fba4fbeb9937 original_time_created = 2024-11-28 01:53:03.159475 UTC target_id = 67ac247f-c934-4c78-857a-fba4fbeb9937 time_created = 2024-11-28 01:53:03.159475 UTC
14032024-11-28T02:01:20.502ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 24.404954ms iteration = 50 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"67ac247f-c934-4c78-857a-fba4fbeb9937","time_created":"2024-11-28T01:53:03.159475Z"}
14042024-11-28T02:01:20.509ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 30.629882ms iteration = 50 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
14052024-11-28T02:01:21.053ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 3a29dc74-08f2-4bfd-9066-d022faa3b6ad roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
14062024-11-28T02:01:21.053ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 3a29dc74-08f2-4bfd-9066-d022faa3b6ad resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
14072024-11-28T02:01:21.055ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2156 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 3a29dc74-08f2-4bfd-9066-d022faa3b6ad response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
14082024-11-28T02:01:22.074ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = cc1c2ff7-326b-4ad5-8517-3e37dbf76584 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
14092024-11-28T02:01:22.074ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = cc1c2ff7-326b-4ad5-8517-3e37dbf76584 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
14102024-11-28T02:01:22.076ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1851 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = cc1c2ff7-326b-4ad5-8517-3e37dbf76584 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
14112024-11-28T02:01:23.097ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 2e469b93-30ed-466d-bd3b-144bc437591d roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
14122024-11-28T02:01:23.097ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 2e469b93-30ed-466d-bd3b-144bc437591d resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
14132024-11-28T02:01:23.099ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2399 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 2e469b93-30ed-466d-bd3b-144bc437591d response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
14142024-11-28T02:01:24.118ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 6be85b46-878b-484d-8613-719693f2ab9c roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
14152024-11-28T02:01:24.119ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 6be85b46-878b-484d-8613-719693f2ab9c resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
14162024-11-28T02:01:24.121ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2327 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 6be85b46-878b-484d-8613-719693f2ab9c response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
14172024-11-28T02:01:24.185ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:55495 req_id = 31123756-837b-44c7-b573-d5f3e8222a82 roles = RoleSet { roles: {} } uri = /metrics/collectors/119eed64-2e24-4d5f-8c42-be6cd8bec8b5/producers?sort_by=id_ascending
14182024-11-28T02:01:24.185ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:55495 req_id = 31123756-837b-44c7-b573-d5f3e8222a82 resource = Database result = Ok(()) uri = /metrics/collectors/119eed64-2e24-4d5f-8c42-be6cd8bec8b5/producers?sort_by=id_ascending
14192024-11-28T02:01:24.188ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 3366 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:55495 req_id = 31123756-837b-44c7-b573-d5f3e8222a82 response_code = 200 uri = /metrics/collectors/119eed64-2e24-4d5f-8c42-be6cd8bec8b5/producers?sort_by=id_ascending
14202024-11-28T02:01:24.189ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:55495 req_id = fb30ca5e-4bdf-4c4f-bf89-a852788462bc roles = RoleSet { roles: {} } uri = /metrics/collectors/119eed64-2e24-4d5f-8c42-be6cd8bec8b5/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJjNDM0YjQ0OC01MTk5LTQ2NmUtYWYzMi00ZjYyOWUzMWM3YjEifX0%3D
14212024-11-28T02:01:24.189ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:55495 req_id = fb30ca5e-4bdf-4c4f-bf89-a852788462bc resource = Database result = Ok(()) uri = /metrics/collectors/119eed64-2e24-4d5f-8c42-be6cd8bec8b5/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJjNDM0YjQ0OC01MTk5LTQ2NmUtYWYzMi00ZjYyOWUzMWM3YjEifX0%3D
14222024-11-28T02:01:24.191ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2200 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:55495 req_id = fb30ca5e-4bdf-4c4f-bf89-a852788462bc response_code = 200 uri = /metrics/collectors/119eed64-2e24-4d5f-8c42-be6cd8bec8b5/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJjNDM0YjQ0OC01MTk5LTQ2NmUtYWYzMi00ZjYyOWUzMWM3YjEifX0%3D
14232024-11-28T02:01:24.193ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1308 local_addr = [fd00:1122:3344:101::b]:55621 method = GET remote_addr = [fd00:1122:3344:101::d]:56417 req_id = 2e60192c-b6a4-4109-80b8-47f078e2e47d response_code = 200 uri = /2469a748-bcd2-4e0d-99cc-f79def0452cc
14242024-11-28T02:01:25.140ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = dabb4034-6428-404b-b957-8753b28f7bef roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
14252024-11-28T02:01:25.141ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = dabb4034-6428-404b-b957-8753b28f7bef resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
14262024-11-28T02:01:25.142ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2291 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = dabb4034-6428-404b-b957-8753b28f7bef response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
14272024-11-28T02:01:26.162ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 1b45fff4-492d-40e4-8b7d-6c59e677a675 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
14282024-11-28T02:01:26.163ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 1b45fff4-492d-40e4-8b7d-6c59e677a675 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
14292024-11-28T02:01:26.164ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2012 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 1b45fff4-492d-40e4-8b7d-6c59e677a675 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
14302024-11-28T02:01:27.183ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = cb096749-4a67-42ba-acf5-8447ee684ff0 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
14312024-11-28T02:01:27.184ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = cb096749-4a67-42ba-acf5-8447ee684ff0 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
14322024-11-28T02:01:27.186ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2580 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = cb096749-4a67-42ba-acf5-8447ee684ff0 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
14332024-11-28T02:01:28.206ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = b4fa6cfc-6619-4929-b8f4-9c923259566a roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
14342024-11-28T02:01:28.206ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = b4fa6cfc-6619-4929-b8f4-9c923259566a resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
14352024-11-28T02:01:28.208ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2259 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = b4fa6cfc-6619-4929-b8f4-9c923259566a response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
14362024-11-28T02:01:29.226ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 1a0eb16b-014b-4ab5-aabe-9cb7541de25e roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
14372024-11-28T02:01:29.226ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 1a0eb16b-014b-4ab5-aabe-9cb7541de25e resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
14382024-11-28T02:01:29.228ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1876 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 1a0eb16b-014b-4ab5-aabe-9cb7541de25e response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
14392024-11-28T02:01:30.248ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 021035dd-1c5c-4e25-9ab3-2811e815abd1 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
14402024-11-28T02:01:30.248ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 021035dd-1c5c-4e25-9ab3-2811e815abd1 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
14412024-11-28T02:01:30.250ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2145 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 021035dd-1c5c-4e25-9ab3-2811e815abd1 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
14422024-11-28T02:01:30.478ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 51 reason = Timeout
14432024-11-28T02:01:30.478ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
14442024-11-28T02:01:30.478ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 51 reason = Timeout
14452024-11-28T02:01:30.478ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
14462024-11-28T02:01:30.478ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
14472024-11-28T02:01:30.478ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
14482024-11-28T02:01:30.480ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
14492024-11-28T02:01:30.480ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
14502024-11-28T02:01:30.480ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14512024-11-28T02:01:30.481ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.591352ms iteration = 51 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
14522024-11-28T02:01:30.483ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
14532024-11-28T02:01:30.483ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
14542024-11-28T02:01:30.483ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
14552024-11-28T02:01:30.484ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
14562024-11-28T02:01:30.485ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
14572024-11-28T02:01:30.486ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14582024-11-28T02:01:30.489ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 67ac247f-c934-4c78-857a-fba4fbeb9937, lookup_type: ById(67ac247f-c934-4c78-857a-fba4fbeb9937) } result = Ok(())
14592024-11-28T02:01:30.489ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
14602024-11-28T02:01:30.489ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
14612024-11-28T02:01:30.500ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 67ac247f-c934-4c78-857a-fba4fbeb9937 original_time_created = 2024-11-28 01:53:03.159475 UTC target_id = 67ac247f-c934-4c78-857a-fba4fbeb9937 time_created = 2024-11-28 01:53:03.159475 UTC
14622024-11-28T02:01:30.500ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 22.201085ms iteration = 51 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"67ac247f-c934-4c78-857a-fba4fbeb9937","time_created":"2024-11-28T01:53:03.159475Z"}
14632024-11-28T02:01:31.268ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 17c09728-9e44-492b-9c33-9872889c4b5f roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
14642024-11-28T02:01:31.269ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 17c09728-9e44-492b-9c33-9872889c4b5f resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
14652024-11-28T02:01:31.270ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2112 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 17c09728-9e44-492b-9c33-9872889c4b5f response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
14662024-11-28T02:01:32.289ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = b443bb9d-6203-4ca6-987c-7418b5b1a5eb roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
14672024-11-28T02:01:32.289ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = b443bb9d-6203-4ca6-987c-7418b5b1a5eb resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
14682024-11-28T02:01:32.291ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2042 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = b443bb9d-6203-4ca6-987c-7418b5b1a5eb response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
14692024-11-28T02:01:33.311ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = b5953f18-7f9b-43e0-987e-3b5ab348cf76 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
14702024-11-28T02:01:33.311ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = b5953f18-7f9b-43e0-987e-3b5ab348cf76 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
14712024-11-28T02:01:33.313ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2574 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = b5953f18-7f9b-43e0-987e-3b5ab348cf76 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
14722024-11-28T02:01:34.194ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1553 local_addr = [fd00:1122:3344:101::b]:55621 method = GET remote_addr = [fd00:1122:3344:101::d]:56417 req_id = bcf98786-ef3d-479f-9453-6e5b8c41706e response_code = 200 uri = /2469a748-bcd2-4e0d-99cc-f79def0452cc
14732024-11-28T02:01:34.332ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 5919ff10-cc9c-4f44-b877-51a9085a1af6 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
14742024-11-28T02:01:34.333ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 5919ff10-cc9c-4f44-b877-51a9085a1af6 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
14752024-11-28T02:01:34.335ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2614 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 5919ff10-cc9c-4f44-b877-51a9085a1af6 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
14762024-11-28T02:01:35.354ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 519934d0-2b1d-41bd-bba9-1ea7ddec4763 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
14772024-11-28T02:01:35.354ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 519934d0-2b1d-41bd-bba9-1ea7ddec4763 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
14782024-11-28T02:01:35.355ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1822 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 519934d0-2b1d-41bd-bba9-1ea7ddec4763 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
14792024-11-28T02:01:36.376ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 74cf3efc-4380-486d-94f8-6306ad7e64d5 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
14802024-11-28T02:01:36.376ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 74cf3efc-4380-486d-94f8-6306ad7e64d5 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
14812024-11-28T02:01:36.378ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1972 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 74cf3efc-4380-486d-94f8-6306ad7e64d5 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
14822024-11-28T02:01:37.397ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 3207d79d-79e2-496c-9cd5-80c8699f9439 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
14832024-11-28T02:01:37.397ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 3207d79d-79e2-496c-9cd5-80c8699f9439 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
14842024-11-28T02:01:37.399ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2182 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 3207d79d-79e2-496c-9cd5-80c8699f9439 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
14852024-11-28T02:01:38.420ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 9627f429-a56d-475e-9641-9f5731484671 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
14862024-11-28T02:01:38.420ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 9627f429-a56d-475e-9641-9f5731484671 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
14872024-11-28T02:01:38.422ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2130 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 9627f429-a56d-475e-9641-9f5731484671 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
14882024-11-28T02:01:39.440ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = fb5ad301-e73e-4430-9a57-5d8d39eed482 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
14892024-11-28T02:01:39.440ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = fb5ad301-e73e-4430-9a57-5d8d39eed482 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
14902024-11-28T02:01:39.442ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1936 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = fb5ad301-e73e-4430-9a57-5d8d39eed482 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
14912024-11-28T02:01:40.460ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 6daede69-3f6e-44c8-9294-801560c906a9 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
14922024-11-28T02:01:40.461ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 6daede69-3f6e-44c8-9294-801560c906a9 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
14932024-11-28T02:01:40.462ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2009 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 6daede69-3f6e-44c8-9294-801560c906a9 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
14942024-11-28T02:01:40.477ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 52 reason = Timeout
14952024-11-28T02:01:40.478ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 18 reason = Timeout
14962024-11-28T02:01:40.478ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
14972024-11-28T02:01:40.478ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
14982024-11-28T02:01:40.478ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 52 reason = Timeout
14992024-11-28T02:01:40.478ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 18 reason = Timeout
15002024-11-28T02:01:40.478ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
15012024-11-28T02:01:40.478ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 18 reason = Timeout
15022024-11-28T02:01:40.478ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
15032024-11-28T02:01:40.478ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
15042024-11-28T02:01:40.478ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
15052024-11-28T02:01:40.478ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 18 reason = Timeout
15062024-11-28T02:01:40.478ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
15072024-11-28T02:01:40.478ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 18 reason = Timeout
15082024-11-28T02:01:40.478ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 18 reason = Timeout
15092024-11-28T02:01:40.478ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
15102024-11-28T02:01:40.478ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
15112024-11-28T02:01:40.478ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 18 reason = Timeout
15122024-11-28T02:01:40.478ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
15132024-11-28T02:01:40.478ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 21 reason = Timeout
15142024-11-28T02:01:40.478ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
15152024-11-28T02:01:40.478ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 18 reason = Timeout
15162024-11-28T02:01:40.478ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
15172024-11-28T02:01:40.478ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 18 reason = Timeout
15182024-11-28T02:01:40.478ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
15192024-11-28T02:01:40.478ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 18 reason = Timeout
15202024-11-28T02:01:40.478ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
15212024-11-28T02:01:40.478ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 18 reason = Timeout
15222024-11-28T02:01:40.478ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
15232024-11-28T02:01:40.478ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 19 reason = Timeout
15242024-11-28T02:01:40.478ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
15252024-11-28T02:01:40.478ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 18 reason = Timeout
15262024-11-28T02:01:40.478ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
15272024-11-28T02:01:40.478ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 18 reason = Timeout
15282024-11-28T02:01:40.478ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
15292024-11-28T02:01:40.478ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
15302024-11-28T02:01:40.478ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
15312024-11-28T02:01:40.478ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
15322024-11-28T02:01:40.478ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
15332024-11-28T02:01:40.478ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
15342024-11-28T02:01:40.478ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
15352024-11-28T02:01:40.478ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
15362024-11-28T02:01:40.478ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
15372024-11-28T02:01:40.478ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
15382024-11-28T02:01:40.478ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
15392024-11-28T02:01:40.478ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
15402024-11-28T02:01:40.478ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
15412024-11-28T02:01:40.478ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
15422024-11-28T02:01:40.478ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
15432024-11-28T02:01:40.478ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
15442024-11-28T02:01:40.478ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
15452024-11-28T02:01:40.480ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 2.723667ms iteration = 18 status = {"errors":[],"garbage_collect_requested":[]}
15462024-11-28T02:01:40.482ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-313847b3-d670-4167-9b09-9e7d3dfcb2a4.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-313847b3-d670-4167-9b09-9e7d3dfcb2a4.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1366, tv_nsec: 952714823 } })
15472024-11-28T02:01:40.482ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1103
15482024-11-28T02:01:40.482ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
15492024-11-28T02:01:40.482ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 4.494508ms iteration = 18 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
15502024-11-28T02:01:40.482ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
15512024-11-28T02:01:40.483ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15522024-11-28T02:01:40.483ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
15532024-11-28T02:01:40.483ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
15542024-11-28T02:01:40.483ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
15552024-11-28T02:01:40.484ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
15562024-11-28T02:01:40.484ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15572024-11-28T02:01:40.485ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15582024-11-28T02:01:40.485ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
15592024-11-28T02:01:40.486ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
15602024-11-28T02:01:40.486ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
15612024-11-28T02:01:40.488ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15622024-11-28T02:01:40.488ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
15632024-11-28T02:01:40.488ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
15642024-11-28T02:01:40.488ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
15652024-11-28T02:01:40.488ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
15662024-11-28T02:01:40.488ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15672024-11-28T02:01:40.488ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
15682024-11-28T02:01:40.488ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
15692024-11-28T02:01:40.490ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15702024-11-28T02:01:40.491ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-313847b3-d670-4167-9b09-9e7d3dfcb2a4.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-313847b3-d670-4167-9b09-9e7d3dfcb2a4.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1366, tv_nsec: 962083336 } })
15712024-11-28T02:01:40.491ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1103
15722024-11-28T02:01:40.492ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
15732024-11-28T02:01:40.492ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
15742024-11-28T02:01:40.492ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
15752024-11-28T02:01:40.492ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
15762024-11-28T02:01:40.492ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
15772024-11-28T02:01:40.492ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
15782024-11-28T02:01:40.492ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
15792024-11-28T02:01:40.493ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
15802024-11-28T02:01:40.493ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
15812024-11-28T02:01:40.498ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
15822024-11-28T02:01:40.498ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
15832024-11-28T02:01:40.500ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15842024-11-28T02:01:40.502ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
15852024-11-28T02:01:40.502ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
15862024-11-28T02:01:40.502ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
15872024-11-28T02:01:40.502ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
15882024-11-28T02:01:40.503ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-313847b3-d670-4167-9b09-9e7d3dfcb2a4.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-313847b3-d670-4167-9b09-9e7d3dfcb2a4.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1366, tv_nsec: 974042803 } })
15892024-11-28T02:01:40.503ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1103 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc
15902024-11-28T02:01:40.504ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 67ac247f-c934-4c78-857a-fba4fbeb9937, lookup_type: ById(67ac247f-c934-4c78-857a-fba4fbeb9937) } result = Ok(())
15912024-11-28T02:01:40.504ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
15922024-11-28T02:01:40.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
15932024-11-28T02:01:40.507ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15942024-11-28T02:01:40.510ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
15952024-11-28T02:01:40.510ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
15962024-11-28T02:01:40.510ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
15972024-11-28T02:01:40.522ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 67ac247f-c934-4c78-857a-fba4fbeb9937 original_time_created = 2024-11-28 01:53:03.159475 UTC target_id = 67ac247f-c934-4c78-857a-fba4fbeb9937 time_created = 2024-11-28 01:53:03.159475 UTC
15982024-11-28T02:01:40.522ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 44.143327ms iteration = 52 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"67ac247f-c934-4c78-857a-fba4fbeb9937","time_created":"2024-11-28T01:53:03.159475Z"}
15992024-11-28T02:01:40.522ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 44.212318ms iteration = 21 status = {"physical_disks_added":0}
16002024-11-28T02:01:40.529ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 51.333062ms iteration = 18 status = {"errors":[],"records_set_to_done":[]}
16012024-11-28T02:01:40.529ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
16022024-11-28T02:01:40.530ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
16032024-11-28T02:01:40.531ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
16042024-11-28T02:01:40.531ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
16052024-11-28T02:01:40.533ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 54.963804ms iteration = 18 status = {"errors":[],"requests_created_ok":[],"start_invoked_ok":[]}
16062024-11-28T02:01:40.535ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
16072024-11-28T02:01:40.535ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
16082024-11-28T02:01:40.536ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 58.71001ms iteration = 18 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
16092024-11-28T02:01:40.537ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
16102024-11-28T02:01:40.537ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
16112024-11-28T02:01:40.539ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
16122024-11-28T02:01:40.539ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
16132024-11-28T02:01:40.540ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
16142024-11-28T02:01:40.540ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
16152024-11-28T02:01:40.540ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
16162024-11-28T02:01:40.540ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
16172024-11-28T02:01:40.540ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 62.873052ms iteration = 52 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
16182024-11-28T02:01:40.541ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16192024-11-28T02:01:40.544ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16202024-11-28T02:01:40.546ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 313847b3-d670-4167-9b09-9e7d3dfcb2a4, lookup_type: ById(313847b3-d670-4167-9b09-9e7d3dfcb2a4) } result = Ok(())
16212024-11-28T02:01:40.546ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
16222024-11-28T02:01:40.546ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
16232024-11-28T02:01:40.546ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
16242024-11-28T02:01:40.546ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
16252024-11-28T02:01:40.546ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
16262024-11-28T02:01:40.547ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
16272024-11-28T02:01:40.547ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 69.838293ms iteration = 18 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
16282024-11-28T02:01:40.548ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 70.296921ms iteration = 18 status = 0
16292024-11-28T02:01:40.548ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
16302024-11-28T02:01:40.548ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
16312024-11-28T02:01:40.549ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "0476410e-819f-4ac9-b003-df1a1cca6499", "content-length": "26", "date": "Thu, 28 Nov 2024 02:01:40 GMT"} })
16322024-11-28T02:01:40.549ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
16332024-11-28T02:01:40.550ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
16342024-11-28T02:01:40.565ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
16352024-11-28T02:01:40.565ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
16362024-11-28T02:01:40.570ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 92.699991ms iteration = 18 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[]}
16372024-11-28T02:01:40.571ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
16382024-11-28T02:01:40.571ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
16392024-11-28T02:01:40.571ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "b6830383-ce4b-4245-85c3-f6f149c00a3e", "content-length": "26", "date": "Thu, 28 Nov 2024 02:01:40 GMT"} })
16402024-11-28T02:01:40.571ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
16412024-11-28T02:01:40.573ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
16422024-11-28T02:01:40.578ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
16432024-11-28T02:01:40.578ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
16442024-11-28T02:01:40.578ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 100.482076ms iteration = 18 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
16452024-11-28T02:01:40.590ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
16462024-11-28T02:01:40.591ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
16472024-11-28T02:01:40.595ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1111 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc zone_address = fd00:1122:3344:101::2
16482024-11-28T02:01:40.595ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
16492024-11-28T02:01:40.595ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "b42acc3c-6c6f-40dd-9b43-290d51e439b5", "content-length": "26", "date": "Thu, 28 Nov 2024 02:01:40 GMT"} })
16502024-11-28T02:01:40.595ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1116 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
16512024-11-28T02:01:40.597ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc
16522024-11-28T02:01:40.628ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
16532024-11-28T02:01:40.629ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
16542024-11-28T02:01:40.630ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16552024-11-28T02:01:40.632ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
16562024-11-28T02:01:40.632ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
16572024-11-28T02:01:40.633ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
16582024-11-28T02:01:40.634ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16592024-11-28T02:01:40.636ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
16602024-11-28T02:01:40.636ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
16612024-11-28T02:01:40.636ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
16622024-11-28T02:01:40.642ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 164.410766ms iteration = 18 status = {}
16632024-11-28T02:01:40.673ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
16642024-11-28T02:01:40.674ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "0d0ea626-c55c-4b66-a73c-738cff7a3e78", "content-length": "1", "date": "Thu, 28 Nov 2024 02:01:40 GMT"} })
16652024-11-28T02:01:40.676ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
16662024-11-28T02:01:40.676ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
16672024-11-28T02:01:40.707ZINFOnexus (BackgroundTasks): VPC route manager sled 313847b3-d670-4167-9b09-9e7d3dfcb2a4 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
16682024-11-28T02:01:40.707ZDEBGnexus (BackgroundTasks): client request SledAgent = 313847b3-d670-4167-9b09-9e7d3dfcb2a4 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
16692024-11-28T02:01:40.709ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 231.711735ms iteration = 18 status = null
16702024-11-28T02:01:40.779ZDEBGnexus (BackgroundTasks): client request SledAgent = 313847b3-d670-4167-9b09-9e7d3dfcb2a4 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
16712024-11-28T02:01:40.802ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
16722024-11-28T02:01:40.802ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
16732024-11-28T02:01:40.804ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 4bb2986f-2e92-4ac3-9dc1-591b6bb60b3e, rack_id: 92a8ff6e-21d3-4c28-bcad-e323847985dc, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(ea300318-9d62-42ed-80ea-21c86590a166) } rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc switch_location = Switch0
16742024-11-28T02:01:40.804ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
16752024-11-28T02:01:40.804ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
16762024-11-28T02:01:40.815ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
16772024-11-28T02:01:40.815ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "79b2a40a-a080-44dc-9db5-9b3ff954cb8a", "content-length": "134", "date": "Thu, 28 Nov 2024 02:01:40 GMT"} })
16782024-11-28T02:01:40.815ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1746 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
16792024-11-28T02:01:40.815ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1773 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = ea300318-9d62-42ed-80ea-21c86590a166
16802024-11-28T02:01:40.815ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
16812024-11-28T02:01:40.815ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
16822024-11-28T02:01:40.815ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
16832024-11-28T02:01:40.815ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
16842024-11-28T02:01:40.815ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
16852024-11-28T02:01:40.815ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1920 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
16862024-11-28T02:01:40.816ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
16872024-11-28T02:01:40.816ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
16882024-11-28T02:01:40.816ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc
16892024-11-28T02:01:40.816ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
16902024-11-28T02:01:40.816ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
16912024-11-28T02:01:40.818ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
16922024-11-28T02:01:40.819ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "8bfb45e4-ccaa-4147-bf0c-3aec3581dd8b", "content-length": "2", "date": "Thu, 28 Nov 2024 02:01:40 GMT"} })
16932024-11-28T02:01:40.819ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
16942024-11-28T02:01:40.819ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "c6264c29-9979-4d87-ad58-6d05591dd06f", "content-length": "52", "date": "Thu, 28 Nov 2024 02:01:40 GMT"} })
16952024-11-28T02:01:40.819ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc switch_location = Switch0
16962024-11-28T02:01:40.819ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
16972024-11-28T02:01:40.825ZDEBGnexus (BackgroundTasks): client response SledAgent = 313847b3-d670-4167-9b09-9e7d3dfcb2a4 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "b5225b2a-9051-44e1-9a55-5474e35ec142", "content-length": "372", "date": "Thu, 28 Nov 2024 02:01:40 GMT"} })
16982024-11-28T02:01:40.825ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
16992024-11-28T02:01:40.826ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
17002024-11-28T02:01:40.829ZDEBGnexus (BackgroundTasks): client response SledAgent = 313847b3-d670-4167-9b09-9e7d3dfcb2a4 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "f811da62-4d57-4d3e-9d4a-9161f4d1b8dd", "content-length": "691", "date": "Thu, 28 Nov 2024 02:01:40 GMT"} })
17012024-11-28T02:01:40.829ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
17022024-11-28T02:01:40.829ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
17032024-11-28T02:01:40.829ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
17042024-11-28T02:01:40.830ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 352.92841ms iteration = 18 status = {}
17052024-11-28T02:01:40.844ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
17062024-11-28T02:01:40.844ZDEBGnexus (BackgroundTasks): client request SledAgent = 313847b3-d670-4167-9b09-9e7d3dfcb2a4 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
17072024-11-28T02:01:40.844ZDEBGnexus (BackgroundTasks): client response SledAgent = 313847b3-d670-4167-9b09-9e7d3dfcb2a4 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "20689ed3-b6ec-43be-9561-36dad394fc2a", "date": "Thu, 28 Nov 2024 02:01:40 GMT"} })
17082024-11-28T02:01:40.844ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
17092024-11-28T02:01:40.844ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
17102024-11-28T02:01:40.846ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
17112024-11-28T02:01:40.846ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
17122024-11-28T02:01:40.849ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
17132024-11-28T02:01:40.850ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
17142024-11-28T02:01:40.853ZINFOnexus (BackgroundTasks): VPC route manager sled 313847b3-d670-4167-9b09-9e7d3dfcb2a4 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
17152024-11-28T02:01:40.853ZDEBGnexus (BackgroundTasks): client request SledAgent = 313847b3-d670-4167-9b09-9e7d3dfcb2a4 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
17162024-11-28T02:01:40.854ZDEBGnexus (BackgroundTasks): client response SledAgent = 313847b3-d670-4167-9b09-9e7d3dfcb2a4 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "40b794bc-b9ef-4e4a-bc34-837987d2a764", "date": "Thu, 28 Nov 2024 02:01:40 GMT"} })
17172024-11-28T02:01:40.856ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 378.075017ms iteration = 19 status = {}
17182024-11-28T02:01:40.919ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "2791573f-e0cd-4379-9dc9-54091f162beb", "date": "Thu, 28 Nov 2024 02:01:40 GMT"} })
17192024-11-28T02:01:40.919ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc switch = Switch0
17202024-11-28T02:01:40.919ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
17212024-11-28T02:01:40.920ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "649a8827-d75d-42b4-84e9-e88480c36a1c", "content-length": "682", "date": "Thu, 28 Nov 2024 02:01:40 GMT"} })
17222024-11-28T02:01:40.920ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
17232024-11-28T02:01:40.920ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
17242024-11-28T02:01:40.923ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
17252024-11-28T02:01:40.924ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
17262024-11-28T02:01:40.927ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
17272024-11-28T02:01:40.927ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
17282024-11-28T02:01:40.929ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
17292024-11-28T02:01:40.930ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
17302024-11-28T02:01:40.937ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 459.322198ms iteration = 18 status = {}
17312024-11-28T02:01:41.245ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:40402 req_id = 2bc9e1cf-8e8a-4172-87fe-910a1a5e726f roles = RoleSet { roles: {} } uri = /nat/ipv4/changeset/6?limit=100
17322024-11-28T02:01:41.246ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:40402 req_id = 2bc9e1cf-8e8a-4172-87fe-910a1a5e726f resource = Database result = Ok(()) uri = /nat/ipv4/changeset/6?limit=100
17332024-11-28T02:01:41.249ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 3250 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:40402 req_id = 2bc9e1cf-8e8a-4172-87fe-910a1a5e726f response_code = 200 uri = /nat/ipv4/changeset/6?limit=100
17342024-11-28T02:01:41.485ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 8fe6703f-b3c0-4389-8ddd-b0bfa8facc5b roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
17352024-11-28T02:01:41.485ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 8fe6703f-b3c0-4389-8ddd-b0bfa8facc5b resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
17362024-11-28T02:01:41.487ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2284 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 8fe6703f-b3c0-4389-8ddd-b0bfa8facc5b response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
17372024-11-28T02:01:42.507ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 18d7276f-c7ad-4ac5-b3e2-3b3d6713e02f roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
17382024-11-28T02:01:42.507ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 18d7276f-c7ad-4ac5-b3e2-3b3d6713e02f resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
17392024-11-28T02:01:42.508ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1741 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 18d7276f-c7ad-4ac5-b3e2-3b3d6713e02f response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
17402024-11-28T02:01:43.527ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 7b6c7fbe-1734-4e21-aed6-361df892f6f2 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
17412024-11-28T02:01:43.528ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 7b6c7fbe-1734-4e21-aed6-361df892f6f2 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
17422024-11-28T02:01:43.530ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2683 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 7b6c7fbe-1734-4e21-aed6-361df892f6f2 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
17432024-11-28T02:01:44.194ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1229 local_addr = [fd00:1122:3344:101::b]:55621 method = GET remote_addr = [fd00:1122:3344:101::d]:56417 req_id = 950df3e9-e20d-4c33-b9f2-774ee4a725d1 response_code = 200 uri = /2469a748-bcd2-4e0d-99cc-f79def0452cc
17442024-11-28T02:01:44.550ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 1ba133a4-8718-4d6a-a20a-6b1b3e703a0c roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
17452024-11-28T02:01:44.550ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 1ba133a4-8718-4d6a-a20a-6b1b3e703a0c resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
17462024-11-28T02:01:44.553ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 3188 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 1ba133a4-8718-4d6a-a20a-6b1b3e703a0c response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
17472024-11-28T02:01:45.580ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 6b9d8ab0-4a59-432b-a449-19938a3edc6b roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
17482024-11-28T02:01:45.580ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 6b9d8ab0-4a59-432b-a449-19938a3edc6b resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
17492024-11-28T02:01:45.582ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2692 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 6b9d8ab0-4a59-432b-a449-19938a3edc6b response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
17502024-11-28T02:01:46.601ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 333c354e-7a4a-4778-9bf8-d528fbad1a30 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
17512024-11-28T02:01:46.601ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 333c354e-7a4a-4778-9bf8-d528fbad1a30 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
17522024-11-28T02:01:46.603ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2121 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 333c354e-7a4a-4778-9bf8-d528fbad1a30 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
17532024-11-28T02:01:47.623ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = cd46420b-99b4-49c3-b7a1-8b329055f967 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
17542024-11-28T02:01:47.623ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = cd46420b-99b4-49c3-b7a1-8b329055f967 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
17552024-11-28T02:01:47.624ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1698 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = cd46420b-99b4-49c3-b7a1-8b329055f967 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
17562024-11-28T02:01:48.642ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = ce03bdec-63f8-42dd-93d1-2142ae54f558 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
17572024-11-28T02:01:48.643ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = ce03bdec-63f8-42dd-93d1-2142ae54f558 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
17582024-11-28T02:01:48.644ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1786 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = ce03bdec-63f8-42dd-93d1-2142ae54f558 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
17592024-11-28T02:01:49.663ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 408df259-dcea-4521-8bfd-f2029a3a1e16 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
17602024-11-28T02:01:49.663ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 408df259-dcea-4521-8bfd-f2029a3a1e16 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
17612024-11-28T02:01:49.665ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1626 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 408df259-dcea-4521-8bfd-f2029a3a1e16 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
17622024-11-28T02:01:50.478ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 53 reason = Timeout
17632024-11-28T02:01:50.478ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 53 reason = Timeout
17642024-11-28T02:01:50.478ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
17652024-11-28T02:01:50.478ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
17662024-11-28T02:01:50.478ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
17672024-11-28T02:01:50.478ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
17682024-11-28T02:01:50.480ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17692024-11-28T02:01:50.480ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
17702024-11-28T02:01:50.480ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
17712024-11-28T02:01:50.482ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 4.00605ms iteration = 53 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
17722024-11-28T02:01:50.482ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
17732024-11-28T02:01:50.482ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
17742024-11-28T02:01:50.482ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
17752024-11-28T02:01:50.484ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
17762024-11-28T02:01:50.485ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
17772024-11-28T02:01:50.486ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17782024-11-28T02:01:50.489ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 67ac247f-c934-4c78-857a-fba4fbeb9937, lookup_type: ById(67ac247f-c934-4c78-857a-fba4fbeb9937) } result = Ok(())
17792024-11-28T02:01:50.489ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
17802024-11-28T02:01:50.489ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
17812024-11-28T02:01:50.497ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 67ac247f-c934-4c78-857a-fba4fbeb9937 original_time_created = 2024-11-28 01:53:03.159475 UTC target_id = 67ac247f-c934-4c78-857a-fba4fbeb9937 time_created = 2024-11-28 01:53:03.159475 UTC
17822024-11-28T02:01:50.497ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 19.464708ms iteration = 53 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"67ac247f-c934-4c78-857a-fba4fbeb9937","time_created":"2024-11-28T01:53:03.159475Z"}
17832024-11-28T02:01:50.684ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 6155941b-8c15-42e9-b410-01461cd45cf4 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
17842024-11-28T02:01:50.684ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 6155941b-8c15-42e9-b410-01461cd45cf4 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
17852024-11-28T02:01:50.685ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1936 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 6155941b-8c15-42e9-b410-01461cd45cf4 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
17862024-11-28T02:01:51.704ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 2fc2e061-4ccd-46b5-8479-8ca10166e4b1 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
17872024-11-28T02:01:51.705ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 2fc2e061-4ccd-46b5-8479-8ca10166e4b1 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
17882024-11-28T02:01:51.706ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1991 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 2fc2e061-4ccd-46b5-8479-8ca10166e4b1 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
17892024-11-28T02:01:52.726ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 18d88a6f-d572-4a98-86e9-9be140f0ed93 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
17902024-11-28T02:01:52.726ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 18d88a6f-d572-4a98-86e9-9be140f0ed93 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
17912024-11-28T02:01:52.729ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2862 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 18d88a6f-d572-4a98-86e9-9be140f0ed93 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
17922024-11-28T02:01:53.746ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = a6d59330-d71d-4507-86b2-58cd7ee281ce roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
17932024-11-28T02:01:53.747ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = a6d59330-d71d-4507-86b2-58cd7ee281ce resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
17942024-11-28T02:01:53.749ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2668 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = a6d59330-d71d-4507-86b2-58cd7ee281ce response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
17952024-11-28T02:01:54.194ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1272 local_addr = [fd00:1122:3344:101::b]:55621 method = GET remote_addr = [fd00:1122:3344:101::d]:56417 req_id = cf2e8b48-9c50-46fc-8eae-24a2f0be0ddd response_code = 200 uri = /2469a748-bcd2-4e0d-99cc-f79def0452cc
17962024-11-28T02:01:54.767ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = d3619c54-bcf9-4f78-9a3b-103ea21ac181 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
17972024-11-28T02:01:54.768ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = d3619c54-bcf9-4f78-9a3b-103ea21ac181 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
17982024-11-28T02:01:54.770ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2106 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = d3619c54-bcf9-4f78-9a3b-103ea21ac181 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
17992024-11-28T02:01:55.790ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 1d3fe045-e248-4deb-9238-f7a8b91508f9 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
18002024-11-28T02:01:55.791ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 1d3fe045-e248-4deb-9238-f7a8b91508f9 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
18012024-11-28T02:01:55.793ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2304 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 1d3fe045-e248-4deb-9238-f7a8b91508f9 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
18022024-11-28T02:01:56.811ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 627a67f7-2297-4a46-ba96-c94217ab1e5d roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
18032024-11-28T02:01:56.811ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 627a67f7-2297-4a46-ba96-c94217ab1e5d resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
18042024-11-28T02:01:56.813ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2358 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 627a67f7-2297-4a46-ba96-c94217ab1e5d response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
18052024-11-28T02:01:57.833ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = bb60b45d-ae3a-4470-aa6b-d9baf7b01bb3 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
18062024-11-28T02:01:57.833ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = bb60b45d-ae3a-4470-aa6b-d9baf7b01bb3 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
18072024-11-28T02:01:57.835ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2150 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = bb60b45d-ae3a-4470-aa6b-d9baf7b01bb3 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
18082024-11-28T02:01:58.853ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = a82b8fe6-2d99-4306-8d6e-b0d5a0d06f23 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
18092024-11-28T02:01:58.854ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = a82b8fe6-2d99-4306-8d6e-b0d5a0d06f23 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
18102024-11-28T02:01:58.855ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1779 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = a82b8fe6-2d99-4306-8d6e-b0d5a0d06f23 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
18112024-11-28T02:01:59.875ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 3f6c018c-83da-4092-89fa-7fc1c58670d4 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
18122024-11-28T02:01:59.875ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 3f6c018c-83da-4092-89fa-7fc1c58670d4 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
18132024-11-28T02:01:59.877ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2369 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 3f6c018c-83da-4092-89fa-7fc1c58670d4 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
18142024-11-28T02:02:00.478ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 54 reason = Timeout
18152024-11-28T02:02:00.478ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
18162024-11-28T02:02:00.478ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 54 reason = Timeout
18172024-11-28T02:02:00.478ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
18182024-11-28T02:02:00.478ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
18192024-11-28T02:02:00.478ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
18202024-11-28T02:02:00.479ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
18212024-11-28T02:02:00.480ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
18222024-11-28T02:02:00.480ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18232024-11-28T02:02:00.481ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.338208ms iteration = 54 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
18242024-11-28T02:02:00.483ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
18252024-11-28T02:02:00.483ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
18262024-11-28T02:02:00.483ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
18272024-11-28T02:02:00.484ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
18282024-11-28T02:02:00.485ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
18292024-11-28T02:02:00.486ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18302024-11-28T02:02:00.489ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 67ac247f-c934-4c78-857a-fba4fbeb9937, lookup_type: ById(67ac247f-c934-4c78-857a-fba4fbeb9937) } result = Ok(())
18312024-11-28T02:02:00.489ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
18322024-11-28T02:02:00.489ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
18332024-11-28T02:02:00.497ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 67ac247f-c934-4c78-857a-fba4fbeb9937 original_time_created = 2024-11-28 01:53:03.159475 UTC target_id = 67ac247f-c934-4c78-857a-fba4fbeb9937 time_created = 2024-11-28 01:53:03.159475 UTC
18342024-11-28T02:02:00.497ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 19.088751ms iteration = 54 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"67ac247f-c934-4c78-857a-fba4fbeb9937","time_created":"2024-11-28T01:53:03.159475Z"}
18352024-11-28T02:02:00.896ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = d78284c3-44c1-49df-9b63-bba5a3f2689a roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
18362024-11-28T02:02:00.896ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = d78284c3-44c1-49df-9b63-bba5a3f2689a resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
18372024-11-28T02:02:00.898ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1672 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = d78284c3-44c1-49df-9b63-bba5a3f2689a response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
18382024-11-28T02:02:01.917ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = a8995bb6-7f20-4fd9-b6aa-e160d33c0fe0 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
18392024-11-28T02:02:01.918ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = a8995bb6-7f20-4fd9-b6aa-e160d33c0fe0 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
18402024-11-28T02:02:01.920ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2774 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = a8995bb6-7f20-4fd9-b6aa-e160d33c0fe0 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
18412024-11-28T02:02:02.938ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = b24ad968-5090-4dcd-9340-21369ca3b690 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
18422024-11-28T02:02:02.938ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = b24ad968-5090-4dcd-9340-21369ca3b690 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
18432024-11-28T02:02:02.940ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1912 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = b24ad968-5090-4dcd-9340-21369ca3b690 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
18442024-11-28T02:02:03.961ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 3bc90ee7-989f-4051-bdaa-3362fc467a36 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
18452024-11-28T02:02:03.961ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 3bc90ee7-989f-4051-bdaa-3362fc467a36 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
18462024-11-28T02:02:03.963ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2614 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 3bc90ee7-989f-4051-bdaa-3362fc467a36 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
18472024-11-28T02:02:04.195ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1707 local_addr = [fd00:1122:3344:101::b]:55621 method = GET remote_addr = [fd00:1122:3344:101::d]:56417 req_id = ea7febf3-0adb-4f9a-a3b3-c3d06a7b72db response_code = 200 uri = /2469a748-bcd2-4e0d-99cc-f79def0452cc
18482024-11-28T02:02:04.982ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 03bf951d-c40f-41d9-a210-5ba6577048d9 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
18492024-11-28T02:02:04.983ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 03bf951d-c40f-41d9-a210-5ba6577048d9 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
18502024-11-28T02:02:04.985ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2579 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 03bf951d-c40f-41d9-a210-5ba6577048d9 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
18512024-11-28T02:02:06.003ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 78f9f2aa-1b3c-4af8-9c12-d1e4ce5bd67b roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
18522024-11-28T02:02:06.003ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 78f9f2aa-1b3c-4af8-9c12-d1e4ce5bd67b resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
18532024-11-28T02:02:06.005ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1865 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 78f9f2aa-1b3c-4af8-9c12-d1e4ce5bd67b response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
18542024-11-28T02:02:07.024ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 3c425eca-de3e-4342-9cb2-be79043a6167 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
18552024-11-28T02:02:07.024ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 3c425eca-de3e-4342-9cb2-be79043a6167 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
18562024-11-28T02:02:07.026ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2291 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 3c425eca-de3e-4342-9cb2-be79043a6167 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
18572024-11-28T02:02:08.045ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = ada0d467-1ce7-4646-b8a3-dc818b8d7e79 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
18582024-11-28T02:02:08.045ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = ada0d467-1ce7-4646-b8a3-dc818b8d7e79 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
18592024-11-28T02:02:08.047ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1955 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = ada0d467-1ce7-4646-b8a3-dc818b8d7e79 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
18602024-11-28T02:02:09.065ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 32c3b6fe-33f3-4a1b-88a6-bc0bb93f7513 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
18612024-11-28T02:02:09.065ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 32c3b6fe-33f3-4a1b-88a6-bc0bb93f7513 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
18622024-11-28T02:02:09.067ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2202 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 32c3b6fe-33f3-4a1b-88a6-bc0bb93f7513 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
18632024-11-28T02:02:10.086ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 6ea7936a-3997-47a9-a461-08cb21dca8ea roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
18642024-11-28T02:02:10.086ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 6ea7936a-3997-47a9-a461-08cb21dca8ea resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
18652024-11-28T02:02:10.088ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2215 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 6ea7936a-3997-47a9-a461-08cb21dca8ea response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
18662024-11-28T02:02:10.478ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 12 reason = Timeout
18672024-11-28T02:02:10.478ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 19 reason = Timeout
18682024-11-28T02:02:10.478ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 11 reason = Timeout
18692024-11-28T02:02:10.478ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 19 reason = Timeout
18702024-11-28T02:02:10.478ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 19 reason = Timeout
18712024-11-28T02:02:10.478ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
18722024-11-28T02:02:10.478ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
18732024-11-28T02:02:10.478ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
18742024-11-28T02:02:10.479ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 55 reason = Timeout
18752024-11-28T02:02:10.479ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 19 reason = Timeout
18762024-11-28T02:02:10.479ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
18772024-11-28T02:02:10.479ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
18782024-11-28T02:02:10.479ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 19 reason = Timeout
18792024-11-28T02:02:10.479ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
18802024-11-28T02:02:10.479ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
18812024-11-28T02:02:10.479ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 19 reason = Timeout
18822024-11-28T02:02:10.479ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 19 reason = Timeout
18832024-11-28T02:02:10.479ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
18842024-11-28T02:02:10.479ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
18852024-11-28T02:02:10.479ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 11 reason = Timeout
18862024-11-28T02:02:10.479ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
18872024-11-28T02:02:10.479ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
18882024-11-28T02:02:10.479ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 22 reason = Timeout
18892024-11-28T02:02:10.479ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
18902024-11-28T02:02:10.479ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 10 reason = Timeout
18912024-11-28T02:02:10.479ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2024-11-28 01:52:10.479019579 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
18922024-11-28T02:02:10.479ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 55 reason = Timeout
18932024-11-28T02:02:10.479ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
18942024-11-28T02:02:10.479ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
18952024-11-28T02:02:10.479ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 19 reason = Timeout
18962024-11-28T02:02:10.479ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
18972024-11-28T02:02:10.479ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 10 reason = Timeout
18982024-11-28T02:02:10.479ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
18992024-11-28T02:02:10.479ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 19 reason = Timeout
19002024-11-28T02:02:10.479ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 11 reason = Timeout
19012024-11-28T02:02:10.479ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
19022024-11-28T02:02:10.479ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
19032024-11-28T02:02:10.479ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
19042024-11-28T02:02:10.479ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 10 reason = Timeout
19052024-11-28T02:02:10.479ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:56
19062024-11-28T02:02:10.479ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 19 reason = Timeout
19072024-11-28T02:02:10.479ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
19082024-11-28T02:02:10.479ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
19092024-11-28T02:02:10.479ZDEBGnexus (BackgroundTasks): activating background_task = crdb_node_id_collector iteration = 5 reason = Timeout
19102024-11-28T02:02:10.479ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 19 reason = Timeout
19112024-11-28T02:02:10.479ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
19122024-11-28T02:02:10.479ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
19132024-11-28T02:02:10.479ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 11 reason = Timeout
19142024-11-28T02:02:10.479ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 11 reason = Timeout
19152024-11-28T02:02:10.479ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 10 reason = Timeout
19162024-11-28T02:02:10.479ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
19172024-11-28T02:02:10.479ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 11 reason = Timeout
19182024-11-28T02:02:10.479ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:83 reason = blueprint disabled target_id = 67ac247f-c934-4c78-857a-fba4fbeb9937
19192024-11-28T02:02:10.479ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 12.761\u{b5}s iteration = 11 status = {"enabled":false,"target_id":"67ac247f-c934-4c78-857a-fba4fbeb9937"}
19202024-11-28T02:02:10.479ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 10 reason = Timeout
19212024-11-28T02:02:10.479ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
19222024-11-28T02:02:10.479ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 20 reason = Timeout
19232024-11-28T02:02:10.479ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
19242024-11-28T02:02:10.479ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 12 reason = Timeout
19252024-11-28T02:02:10.479ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
19262024-11-28T02:02:10.479ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 19 reason = Timeout
19272024-11-28T02:02:10.479ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
19282024-11-28T02:02:10.479ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 19 reason = Timeout
19292024-11-28T02:02:10.479ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
19302024-11-28T02:02:10.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
19312024-11-28T02:02:10.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
19322024-11-28T02:02:10.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
19332024-11-28T02:02:10.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
19342024-11-28T02:02:10.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
19352024-11-28T02:02:10.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
19362024-11-28T02:02:10.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
19372024-11-28T02:02:10.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
19382024-11-28T02:02:10.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
19392024-11-28T02:02:10.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
19402024-11-28T02:02:10.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
19412024-11-28T02:02:10.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
19422024-11-28T02:02:10.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
19432024-11-28T02:02:10.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
19442024-11-28T02:02:10.479ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
19452024-11-28T02:02:10.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
19462024-11-28T02:02:10.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
19472024-11-28T02:02:10.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
19482024-11-28T02:02:10.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
19492024-11-28T02:02:10.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
19502024-11-28T02:02:10.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
19512024-11-28T02:02:10.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
19522024-11-28T02:02:10.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
19532024-11-28T02:02:10.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
19542024-11-28T02:02:10.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
19552024-11-28T02:02:10.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
19562024-11-28T02:02:10.479ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("36d4f615-7461-4364-960c-89ece3aaabaf.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("eb274350-c0b0-4bce-9c8a-502880f2c905.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("eb274350-c0b0-4bce-9c8a-502880f2c905.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::8))) }], valid_until: Instant { tv_sec: 1396, tv_nsec: 950458654 } })
19572024-11-28T02:02:10.479ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("41ea4a99-3038-474e-bf28-2ad338f3243f.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("9402a51b-d30f-4544-93ec-0f5dc7a485e4.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("d23bc5f2-6b92-4992-bdd2-8695e5bf9c0f.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("d23bc5f2-6b92-4992-bdd2-8695e5bf9c0f.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:1::1))) }], valid_until: Instant { tv_sec: 1396, tv_nsec: 950458994 } })
19582024-11-28T02:02:10.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
19592024-11-28T02:02:10.480ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
19602024-11-28T02:02:10.480ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
19612024-11-28T02:02:10.480ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
19622024-11-28T02:02:10.480ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
19632024-11-28T02:02:10.480ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
19642024-11-28T02:02:10.480ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } dns_group = external
19652024-11-28T02:02:10.480ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.272522ms iteration = 11 status = {"addresses":["[fd00:1122:3344:101::9]:5353","[fd00:1122:3344:101::8]:5353"]}
19662024-11-28T02:02:10.480ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
19672024-11-28T02:02:10.480ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } dns_group = internal
19682024-11-28T02:02:10.480ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.349683ms iteration = 11 status = {"addresses":["[fd00:1122:3344:3::1]:5353","[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:1::1]:5353"]}
19692024-11-28T02:02:10.482ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19702024-11-28T02:02:10.483ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
19712024-11-28T02:02:10.483ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
19722024-11-28T02:02:10.483ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 4.766223ms iteration = 19 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
19732024-11-28T02:02:10.483ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-313847b3-d670-4167-9b09-9e7d3dfcb2a4.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-313847b3-d670-4167-9b09-9e7d3dfcb2a4.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1396, tv_nsec: 954237859 } })
19742024-11-28T02:02:10.483ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19752024-11-28T02:02:10.483ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1103
19762024-11-28T02:02:10.484ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
19772024-11-28T02:02:10.484ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
19782024-11-28T02:02:10.485ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
19792024-11-28T02:02:10.486ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19802024-11-28T02:02:10.486ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
19812024-11-28T02:02:10.486ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
19822024-11-28T02:02:10.487ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
19832024-11-28T02:02:10.487ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19842024-11-28T02:02:10.487ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
19852024-11-28T02:02:10.488ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
19862024-11-28T02:02:10.489ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 10.40526ms iteration = 19 status = {"errors":[],"records_set_to_done":[]}
19872024-11-28T02:02:10.489ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
19882024-11-28T02:02:10.489ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
19892024-11-28T02:02:10.489ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-313847b3-d670-4167-9b09-9e7d3dfcb2a4.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-313847b3-d670-4167-9b09-9e7d3dfcb2a4.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1396, tv_nsec: 960375926 } })
19902024-11-28T02:02:10.489ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
19912024-11-28T02:02:10.490ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1103
19922024-11-28T02:02:10.490ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
19932024-11-28T02:02:10.490ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
19942024-11-28T02:02:10.490ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
19952024-11-28T02:02:10.491ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2024-11-28 01:52:10.479019579 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
19962024-11-28T02:02:10.491ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 12.850063ms iteration = 10 status = {"expiration":"2024-11-28T01:52:10.479019579Z","pruned":[]}
19972024-11-28T02:02:10.493ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19982024-11-28T02:02:10.493ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19992024-11-28T02:02:10.494ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20002024-11-28T02:02:10.496ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
20012024-11-28T02:02:10.496ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
20022024-11-28T02:02:10.496ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
20032024-11-28T02:02:10.496ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
20042024-11-28T02:02:10.497ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
20052024-11-28T02:02:10.497ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
20062024-11-28T02:02:10.497ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
20072024-11-28T02:02:10.497ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
20082024-11-28T02:02:10.498ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
20092024-11-28T02:02:10.498ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 18.869258ms iteration = 10 status = {"errors":[],"found_port_ok":[]}
20102024-11-28T02:02:10.498ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20112024-11-28T02:02:10.499ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
20122024-11-28T02:02:10.499ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
20132024-11-28T02:02:10.501ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
20142024-11-28T02:02:10.501ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
20152024-11-28T02:02:10.501ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
20162024-11-28T02:02:10.502ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20172024-11-28T02:02:10.504ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
20182024-11-28T02:02:10.504ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
20192024-11-28T02:02:10.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
20202024-11-28T02:02:10.504ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20212024-11-28T02:02:10.507ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
20222024-11-28T02:02:10.507ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
20232024-11-28T02:02:10.507ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
20242024-11-28T02:02:10.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
20252024-11-28T02:02:10.507ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20262024-11-28T02:02:10.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
20272024-11-28T02:02:10.509ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
20282024-11-28T02:02:10.509ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
20292024-11-28T02:02:10.509ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
20302024-11-28T02:02:10.509ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
20312024-11-28T02:02:10.509ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
20322024-11-28T02:02:10.516ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20332024-11-28T02:02:10.517ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
20342024-11-28T02:02:10.517ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
20352024-11-28T02:02:10.517ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
20362024-11-28T02:02:10.517ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
20372024-11-28T02:02:10.517ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
20382024-11-28T02:02:10.517ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
20392024-11-28T02:02:10.520ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
20402024-11-28T02:02:10.520ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 41.46138ms iteration = 19 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
20412024-11-28T02:02:10.521ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
20422024-11-28T02:02:10.525ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
20432024-11-28T02:02:10.525ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
20442024-11-28T02:02:10.525ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
20452024-11-28T02:02:10.525ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
20462024-11-28T02:02:10.527ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20472024-11-28T02:02:10.528ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-313847b3-d670-4167-9b09-9e7d3dfcb2a4.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-313847b3-d670-4167-9b09-9e7d3dfcb2a4.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1396, tv_nsec: 999186380 } })
20482024-11-28T02:02:10.529ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1103 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc
20492024-11-28T02:02:10.530ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20502024-11-28T02:02:10.530ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
20512024-11-28T02:02:10.530ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
20522024-11-28T02:02:10.530ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
20532024-11-28T02:02:10.531ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
20542024-11-28T02:02:10.531ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
20552024-11-28T02:02:10.534ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
20562024-11-28T02:02:10.534ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
20572024-11-28T02:02:10.534ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
20582024-11-28T02:02:10.535ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20592024-11-28T02:02:10.536ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20602024-11-28T02:02:10.538ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
20612024-11-28T02:02:10.538ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
20622024-11-28T02:02:10.538ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
20632024-11-28T02:02:10.539ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 67ac247f-c934-4c78-857a-fba4fbeb9937, lookup_type: ById(67ac247f-c934-4c78-857a-fba4fbeb9937) } result = Ok(())
20642024-11-28T02:02:10.539ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
20652024-11-28T02:02:10.539ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
20662024-11-28T02:02:10.540ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 61.086601ms iteration = 19 status = {"errors":[],"garbage_collect_requested":[]}
20672024-11-28T02:02:10.541ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
20682024-11-28T02:02:10.541ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
20692024-11-28T02:02:10.542ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
20702024-11-28T02:02:10.542ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
20712024-11-28T02:02:10.542ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
20722024-11-28T02:02:10.542ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
20732024-11-28T02:02:10.542ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
20742024-11-28T02:02:10.543ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
20752024-11-28T02:02:10.544ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20762024-11-28T02:02:10.546ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20772024-11-28T02:02:10.546ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
20782024-11-28T02:02:10.546ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
20792024-11-28T02:02:10.546ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 67.856069ms iteration = 55 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
20802024-11-28T02:02:10.546ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
20812024-11-28T02:02:10.547ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20822024-11-28T02:02:10.548ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
20832024-11-28T02:02:10.549ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
20842024-11-28T02:02:10.550ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
20852024-11-28T02:02:10.550ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
20862024-11-28T02:02:10.550ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:171 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
20872024-11-28T02:02:10.550ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 71.332849ms iteration = 10 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
20882024-11-28T02:02:10.551ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
20892024-11-28T02:02:10.551ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
20902024-11-28T02:02:10.551ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
20912024-11-28T02:02:10.551ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
20922024-11-28T02:02:10.551ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
20932024-11-28T02:02:10.551ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
20942024-11-28T02:02:10.551ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
20952024-11-28T02:02:10.552ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
20962024-11-28T02:02:10.555ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 75.99667ms iteration = 19 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
20972024-11-28T02:02:10.555ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20982024-11-28T02:02:10.555ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20992024-11-28T02:02:10.555ZDEBGnexus (BackgroundTasks): activation complete background_task = crdb_node_id_collector elapsed = 76.814044ms iteration = 5 status = {"nsuccess":5}
21002024-11-28T02:02:10.557ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
21012024-11-28T02:02:10.557ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
21022024-11-28T02:02:10.558ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
21032024-11-28T02:02:10.560ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
21042024-11-28T02:02:10.560ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:140 found = 0 not_ready_to_be_deleted = 0
21052024-11-28T02:02:10.560ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 80.994276ms iteration = 10 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
21062024-11-28T02:02:10.560ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
21072024-11-28T02:02:10.560ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
21082024-11-28T02:02:10.560ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
21092024-11-28T02:02:10.562ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 67ac247f-c934-4c78-857a-fba4fbeb9937 original_time_created = 2024-11-28 01:53:03.159475 UTC target_id = 67ac247f-c934-4c78-857a-fba4fbeb9937 time_created = 2024-11-28 01:53:03.159475 UTC
21102024-11-28T02:02:10.562ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 83.390208ms iteration = 55 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"67ac247f-c934-4c78-857a-fba4fbeb9937","time_created":"2024-11-28T01:53:03.159475Z"}
21112024-11-28T02:02:10.569ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
21122024-11-28T02:02:10.569ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
21132024-11-28T02:02:10.569ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
21142024-11-28T02:02:10.569ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
21152024-11-28T02:02:10.570ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
21162024-11-28T02:02:10.571ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21172024-11-28T02:02:10.575ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
21182024-11-28T02:02:10.575ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
21192024-11-28T02:02:10.575ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
21202024-11-28T02:02:10.575ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
21212024-11-28T02:02:10.576ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
21222024-11-28T02:02:10.577ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
21232024-11-28T02:02:10.577ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
21242024-11-28T02:02:10.577ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 98.317038ms iteration = 19 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
21252024-11-28T02:02:10.577ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
21262024-11-28T02:02:10.577ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 808a324e-c913-4bfc-91a5-d43c961cd511 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
21272024-11-28T02:02:10.577ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
21282024-11-28T02:02:10.578ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
21292024-11-28T02:02:10.578ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
21302024-11-28T02:02:10.578ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
21312024-11-28T02:02:10.579ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
21322024-11-28T02:02:10.579ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
21332024-11-28T02:02:10.579ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21342024-11-28T02:02:10.580ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21352024-11-28T02:02:10.581ZWARNnexus (BackgroundTasks): instance reincarnation completed with errors background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:103 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0 query_errors = 0 restart_errors = 0
21362024-11-28T02:02:10.581ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 102.634042ms iteration = 10 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
21372024-11-28T02:02:10.581ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 102.831126ms iteration = 19 status = {"errors":[],"requests_created_ok":[],"start_invoked_ok":[]}
21382024-11-28T02:02:10.582ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
21392024-11-28T02:02:10.583ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
21402024-11-28T02:02:10.583ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
21412024-11-28T02:02:10.583ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 313847b3-d670-4167-9b09-9e7d3dfcb2a4, lookup_type: ById(313847b3-d670-4167-9b09-9e7d3dfcb2a4) } result = Ok(())
21422024-11-28T02:02:10.583ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
21432024-11-28T02:02:10.583ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
21442024-11-28T02:02:10.585ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 808a324e-c913-4bfc-91a5-d43c961cd511 dns_zone_name = control-plane.oxide.internal found_so_far = 51 version = 1
21452024-11-28T02:02:10.585ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 106.265046ms iteration = 19 status = 0
21462024-11-28T02:02:10.585ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
21472024-11-28T02:02:10.585ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2024-11-28 01:53:42.502562 UTC dns_group = internal generation = 1
21482024-11-28T02:02:10.585ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 106.306877ms iteration = 11 status = {"generation":1}
21492024-11-28T02:02:10.586ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 107.66579ms iteration = 19 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[]}
21502024-11-28T02:02:10.601ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2024-11-28T01:53:08.890632Z, time_modified: 2024-11-28T01:53:08.890632Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 7e159834-997a-4c03-b9db-1e91d783221c, db_silo: Silo { identity: SiloIdentity { id: 7e159834-997a-4c03-b9db-1e91d783221c, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-11-28T01:53:43.308696Z, time_modified: 2024-11-28T01:53:43.308696Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "bc39d1b3b4bb84a49d6458ed7aedc5bca9e77742c7b35882b64d188c0602d306" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 7e159834-997a-4c03-b9db-1e91d783221c, db_silo: Silo { identity: SiloIdentity { id: 7e159834-997a-4c03-b9db-1e91d783221c, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-11-28T01:53:43.308696Z, time_modified: 2024-11-28T01:53:43.308696Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "bc39d1b3b4bb84a49d6458ed7aedc5bca9e77742c7b35882b64d188c0602d306" }] }) }
21512024-11-28T02:02:10.601ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 122.512038ms iteration = 11 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"7e159834-997a-4c03-b9db-1e91d783221c","tls_certs":["bc39d1b3b4bb84a49d6458ed7aedc5bca9e77742c7b35882b64d188c0602d306"]}},"default_endpoint":{"silo_id":"7e159834-997a-4c03-b9db-1e91d783221c","tls_certs":["bc39d1b3b4bb84a49d6458ed7aedc5bca9e77742c7b35882b64d188c0602d306"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
21522024-11-28T02:02:10.606ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
21532024-11-28T02:02:10.606ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = a7e6af61-408c-483d-b043-c27ccf55708e dns_zone_name = oxide.test found_so_far = 0 version = 2
21542024-11-28T02:02:10.606ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
21552024-11-28T02:02:10.606ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
21562024-11-28T02:02:10.608ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21572024-11-28T02:02:10.613ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
21582024-11-28T02:02:10.613ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
21592024-11-28T02:02:10.613ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
21602024-11-28T02:02:10.615ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = a7e6af61-408c-483d-b043-c27ccf55708e dns_zone_name = oxide.test found_so_far = 1 version = 2
21612024-11-28T02:02:10.615ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
21622024-11-28T02:02:10.615ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2024-11-28 01:53:43.351579 UTC dns_group = external generation = 2
21632024-11-28T02:02:10.615ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 136.250157ms iteration = 11 status = {"generation":2}
21642024-11-28T02:02:10.618ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
21652024-11-28T02:02:10.619ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
21662024-11-28T02:02:10.626ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
21672024-11-28T02:02:10.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
21682024-11-28T02:02:10.645ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
21692024-11-28T02:02:10.648ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 169.408853ms iteration = 22 status = {"physical_disks_added":0}
21702024-11-28T02:02:10.651ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
21712024-11-28T02:02:10.651ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
21722024-11-28T02:02:10.652ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "d7c7df6c-97b6-4f23-a679-7b17f7943895", "content-length": "26", "date": "Thu, 28 Nov 2024 02:02:10 GMT"} })
21732024-11-28T02:02:10.652ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
21742024-11-28T02:02:10.654ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
21752024-11-28T02:02:10.705ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1111 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc zone_address = fd00:1122:3344:101::2
21762024-11-28T02:02:10.705ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
21772024-11-28T02:02:10.705ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "49110ad2-eda9-498f-a34b-a2f69b6065da", "content-length": "26", "date": "Thu, 28 Nov 2024 02:02:10 GMT"} })
21782024-11-28T02:02:10.705ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1116 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
21792024-11-28T02:02:10.708ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc
21802024-11-28T02:02:10.772ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
21812024-11-28T02:02:10.841ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
21822024-11-28T02:02:10.863ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 384.704772ms iteration = 19 status = {}
21832024-11-28T02:02:10.878ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
21842024-11-28T02:02:10.878ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
21852024-11-28T02:02:10.878ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "e5622cd5-0306-457e-aa3a-d62899099764", "content-length": "26", "date": "Thu, 28 Nov 2024 02:02:10 GMT"} })
21862024-11-28T02:02:10.878ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
21872024-11-28T02:02:10.880ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
21882024-11-28T02:02:10.987ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
21892024-11-28T02:02:10.987ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "5c1dae4f-18af-49d2-8c94-f62009ef9c5d", "date": "Thu, 28 Nov 2024 02:02:10 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
21902024-11-28T02:02:10.987ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
21912024-11-28T02:02:10.989ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "9fdedd73-8127-4be4-9484-80efe468f7cb", "date": "Thu, 28 Nov 2024 02:02:10 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
21922024-11-28T02:02:10.989ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
21932024-11-28T02:02:10.992ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "15ae0317-3f08-4883-990c-f23dce84cb0a", "date": "Thu, 28 Nov 2024 02:02:10 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
21942024-11-28T02:02:10.992ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
21952024-11-28T02:02:10.994ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
21962024-11-28T02:02:10.994ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 514.946725ms iteration = 12 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
21972024-11-28T02:02:11.031ZINFOnexus (BackgroundTasks): VPC route manager sled 313847b3-d670-4167-9b09-9e7d3dfcb2a4 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
21982024-11-28T02:02:11.031ZDEBGnexus (BackgroundTasks): client request SledAgent = 313847b3-d670-4167-9b09-9e7d3dfcb2a4 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
21992024-11-28T02:02:11.031ZDEBGnexus (BackgroundTasks): client response SledAgent = 313847b3-d670-4167-9b09-9e7d3dfcb2a4 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "625931cc-f5eb-4bd1-8553-4664c8938f29", "content-length": "372", "date": "Thu, 28 Nov 2024 02:02:11 GMT"} })
22002024-11-28T02:02:11.031ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
22012024-11-28T02:02:11.031ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
22022024-11-28T02:02:11.033ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22032024-11-28T02:02:11.035ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
22042024-11-28T02:02:11.035ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
22052024-11-28T02:02:11.035ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
22062024-11-28T02:02:11.037ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22072024-11-28T02:02:11.039ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
22082024-11-28T02:02:11.039ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
22092024-11-28T02:02:11.039ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
22102024-11-28T02:02:11.092ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
22112024-11-28T02:02:11.092ZDEBGnexus (BackgroundTasks): client request SledAgent = 313847b3-d670-4167-9b09-9e7d3dfcb2a4 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
22122024-11-28T02:02:11.092ZDEBGnexus (BackgroundTasks): client response SledAgent = 313847b3-d670-4167-9b09-9e7d3dfcb2a4 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "c0df739a-0685-43c9-b15b-9522db484a8a", "date": "Thu, 28 Nov 2024 02:02:11 GMT"} })
22132024-11-28T02:02:11.092ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
22142024-11-28T02:02:11.093ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
22152024-11-28T02:02:11.094ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
22162024-11-28T02:02:11.094ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
22172024-11-28T02:02:11.094ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
22182024-11-28T02:02:11.094ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "25795fd9-4c47-468b-9329-d5e8ad287f4e", "date": "Thu, 28 Nov 2024 02:02:10 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
22192024-11-28T02:02:11.094ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
22202024-11-28T02:02:11.096ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "e4507b82-6cb7-4f29-b309-8235c6cadff0", "date": "Thu, 28 Nov 2024 02:02:10 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
22212024-11-28T02:02:11.096ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
22222024-11-28T02:02:11.096ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
22232024-11-28T02:02:11.096ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
22242024-11-28T02:02:11.098ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
22252024-11-28T02:02:11.098ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 619.980678ms iteration = 12 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
22262024-11-28T02:02:11.099ZINFOnexus (BackgroundTasks): VPC route manager sled 313847b3-d670-4167-9b09-9e7d3dfcb2a4 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
22272024-11-28T02:02:11.099ZDEBGnexus (BackgroundTasks): client request SledAgent = 313847b3-d670-4167-9b09-9e7d3dfcb2a4 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
22282024-11-28T02:02:11.099ZDEBGnexus (BackgroundTasks): client response SledAgent = 313847b3-d670-4167-9b09-9e7d3dfcb2a4 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "6a2434f5-affa-47bd-8f20-27b4b1551e5b", "date": "Thu, 28 Nov 2024 02:02:11 GMT"} })
22292024-11-28T02:02:11.101ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 622.117396ms iteration = 20 status = {}
22302024-11-28T02:02:11.108ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 30014e61-42df-47c1-a34f-a6cf72374a87 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
22312024-11-28T02:02:11.108ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 30014e61-42df-47c1-a34f-a6cf72374a87 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
22322024-11-28T02:02:11.110ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2146 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 30014e61-42df-47c1-a34f-a6cf72374a87 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
22332024-11-28T02:02:11.121ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
22342024-11-28T02:02:11.121ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "5e67c044-d268-410a-b097-451eabff1fce", "content-length": "1", "date": "Thu, 28 Nov 2024 02:02:11 GMT"} })
22352024-11-28T02:02:11.125ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
22362024-11-28T02:02:11.126ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
22372024-11-28T02:02:11.133ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
22382024-11-28T02:02:11.133ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
22392024-11-28T02:02:11.134ZDEBGnexus (BackgroundTasks): client request SledAgent = 313847b3-d670-4167-9b09-9e7d3dfcb2a4 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
22402024-11-28T02:02:11.134ZDEBGnexus (BackgroundTasks): client response SledAgent = 313847b3-d670-4167-9b09-9e7d3dfcb2a4 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "0b342263-3fee-44c0-820e-48cffd0f4906", "content-length": "691", "date": "Thu, 28 Nov 2024 02:02:11 GMT"} })
22412024-11-28T02:02:11.134ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
22422024-11-28T02:02:11.134ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
22432024-11-28T02:02:11.134ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
22442024-11-28T02:02:11.136ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 656.949301ms iteration = 19 status = {}
22452024-11-28T02:02:11.136ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 657.315747ms iteration = 19 status = null
22462024-11-28T02:02:11.136ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 4bb2986f-2e92-4ac3-9dc1-591b6bb60b3e, rack_id: 92a8ff6e-21d3-4c28-bcad-e323847985dc, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(ea300318-9d62-42ed-80ea-21c86590a166) } rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc switch_location = Switch0
22472024-11-28T02:02:11.136ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
22482024-11-28T02:02:11.137ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
22492024-11-28T02:02:11.146ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
22502024-11-28T02:02:11.146ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "70d9f86d-eebe-481d-b79e-d3e0f926ca66", "content-length": "134", "date": "Thu, 28 Nov 2024 02:02:11 GMT"} })
22512024-11-28T02:02:11.146ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1746 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
22522024-11-28T02:02:11.146ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1773 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = ea300318-9d62-42ed-80ea-21c86590a166
22532024-11-28T02:02:11.146ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
22542024-11-28T02:02:11.146ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
22552024-11-28T02:02:11.146ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
22562024-11-28T02:02:11.146ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
22572024-11-28T02:02:11.146ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
22582024-11-28T02:02:11.146ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1920 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
22592024-11-28T02:02:11.146ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
22602024-11-28T02:02:11.146ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
22612024-11-28T02:02:11.147ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc
22622024-11-28T02:02:11.147ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
22632024-11-28T02:02:11.147ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
22642024-11-28T02:02:11.149ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
22652024-11-28T02:02:11.149ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "50d1df03-0cbe-4c76-85c6-c0d3013c4d08", "content-length": "2", "date": "Thu, 28 Nov 2024 02:02:11 GMT"} })
22662024-11-28T02:02:11.149ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
22672024-11-28T02:02:11.149ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "8428ffa5-9380-42ac-913e-d6205601b4c3", "content-length": "52", "date": "Thu, 28 Nov 2024 02:02:11 GMT"} })
22682024-11-28T02:02:11.149ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc switch_location = Switch0
22692024-11-28T02:02:11.149ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
22702024-11-28T02:02:11.278ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:40402 req_id = eda1f2a4-1a8a-41dd-bae7-959d5886ce00 roles = RoleSet { roles: {} } uri = /nat/ipv4/changeset/6?limit=100
22712024-11-28T02:02:11.279ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:40402 req_id = eda1f2a4-1a8a-41dd-bae7-959d5886ce00 resource = Database result = Ok(()) uri = /nat/ipv4/changeset/6?limit=100
22722024-11-28T02:02:11.282ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 3550 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:40402 req_id = eda1f2a4-1a8a-41dd-bae7-959d5886ce00 response_code = 200 uri = /nat/ipv4/changeset/6?limit=100
22732024-11-28T02:02:11.323ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "5c0b6371-2fd2-44fa-a6a1-0d9bed570fea", "date": "Thu, 28 Nov 2024 02:02:11 GMT"} })
22742024-11-28T02:02:11.323ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc switch = Switch0
22752024-11-28T02:02:11.323ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
22762024-11-28T02:02:11.323ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "5a90d474-7155-43c3-9a18-fcea33fe1f0e", "content-length": "682", "date": "Thu, 28 Nov 2024 02:02:11 GMT"} })
22772024-11-28T02:02:11.323ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
22782024-11-28T02:02:11.324ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
22792024-11-28T02:02:11.328ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
22802024-11-28T02:02:11.328ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
22812024-11-28T02:02:11.332ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
22822024-11-28T02:02:11.332ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
22832024-11-28T02:02:11.334ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
22842024-11-28T02:02:11.334ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
22852024-11-28T02:02:11.342ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 863.459167ms iteration = 19 status = {}
22862024-11-28T02:02:12.133ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 89fc0b02-cd29-4562-a722-d8955e6c6028 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
22872024-11-28T02:02:12.133ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 89fc0b02-cd29-4562-a722-d8955e6c6028 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
22882024-11-28T02:02:12.135ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2078 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 89fc0b02-cd29-4562-a722-d8955e6c6028 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
22892024-11-28T02:02:13.154ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = c9c2ae60-fd9b-4702-8a6b-3e8732896cb4 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
22902024-11-28T02:02:13.154ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = c9c2ae60-fd9b-4702-8a6b-3e8732896cb4 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
22912024-11-28T02:02:13.155ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1535 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = c9c2ae60-fd9b-4702-8a6b-3e8732896cb4 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
22922024-11-28T02:02:14.174ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 500d3dd4-d9a3-4f45-8f0c-54bc26cfe46f roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
22932024-11-28T02:02:14.174ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 500d3dd4-d9a3-4f45-8f0c-54bc26cfe46f resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
22942024-11-28T02:02:14.176ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1867 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 500d3dd4-d9a3-4f45-8f0c-54bc26cfe46f response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
22952024-11-28T02:02:14.194ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1265 local_addr = [fd00:1122:3344:101::b]:55621 method = GET remote_addr = [fd00:1122:3344:101::d]:56417 req_id = acba68d5-c179-44b3-9d55-c594d0632fa8 response_code = 200 uri = /2469a748-bcd2-4e0d-99cc-f79def0452cc
22962024-11-28T02:02:15.196ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = bd03a37a-8ea6-4825-8b28-0637c2f2db24 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
22972024-11-28T02:02:15.196ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = bd03a37a-8ea6-4825-8b28-0637c2f2db24 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
22982024-11-28T02:02:15.197ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1754 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = bd03a37a-8ea6-4825-8b28-0637c2f2db24 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
22992024-11-28T02:02:16.215ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 6e786270-8472-4f65-9412-425f26d5b718 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
23002024-11-28T02:02:16.216ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 6e786270-8472-4f65-9412-425f26d5b718 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
23012024-11-28T02:02:16.217ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1854 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 6e786270-8472-4f65-9412-425f26d5b718 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
23022024-11-28T02:02:17.235ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 298d4761-d793-47c0-9a17-ab56413b941b roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
23032024-11-28T02:02:17.236ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 298d4761-d793-47c0-9a17-ab56413b941b resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
23042024-11-28T02:02:17.237ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2016 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 298d4761-d793-47c0-9a17-ab56413b941b response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
23052024-11-28T02:02:18.257ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 8eb3bc1b-af93-47e5-822f-10f3e4bc6cc2 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
23062024-11-28T02:02:18.258ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 8eb3bc1b-af93-47e5-822f-10f3e4bc6cc2 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
23072024-11-28T02:02:18.259ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1843 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 8eb3bc1b-af93-47e5-822f-10f3e4bc6cc2 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
23082024-11-28T02:02:19.280ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 9b04de09-53a3-4519-bc4e-19c3132f54d5 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
23092024-11-28T02:02:19.280ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 9b04de09-53a3-4519-bc4e-19c3132f54d5 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
23102024-11-28T02:02:19.282ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2370 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 9b04de09-53a3-4519-bc4e-19c3132f54d5 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
23112024-11-28T02:02:20.302ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 64f15740-66e0-4b97-8ab5-fe9a0c07e99a roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
23122024-11-28T02:02:20.302ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 64f15740-66e0-4b97-8ab5-fe9a0c07e99a resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
23132024-11-28T02:02:20.304ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2356 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 64f15740-66e0-4b97-8ab5-fe9a0c07e99a response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
23142024-11-28T02:02:20.479ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 56 reason = Timeout
23152024-11-28T02:02:20.479ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
23162024-11-28T02:02:20.479ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 56 reason = Timeout
23172024-11-28T02:02:20.479ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
23182024-11-28T02:02:20.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
23192024-11-28T02:02:20.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
23202024-11-28T02:02:20.480ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
23212024-11-28T02:02:20.480ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
23222024-11-28T02:02:20.481ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23232024-11-28T02:02:20.482ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.509161ms iteration = 56 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
23242024-11-28T02:02:20.484ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
23252024-11-28T02:02:20.484ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
23262024-11-28T02:02:20.484ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
23272024-11-28T02:02:20.486ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
23282024-11-28T02:02:20.486ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
23292024-11-28T02:02:20.487ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23302024-11-28T02:02:20.490ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 67ac247f-c934-4c78-857a-fba4fbeb9937, lookup_type: ById(67ac247f-c934-4c78-857a-fba4fbeb9937) } result = Ok(())
23312024-11-28T02:02:20.490ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
23322024-11-28T02:02:20.490ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
23332024-11-28T02:02:20.499ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 67ac247f-c934-4c78-857a-fba4fbeb9937 original_time_created = 2024-11-28 01:53:03.159475 UTC target_id = 67ac247f-c934-4c78-857a-fba4fbeb9937 time_created = 2024-11-28 01:53:03.159475 UTC
23342024-11-28T02:02:20.499ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 20.74618ms iteration = 56 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"67ac247f-c934-4c78-857a-fba4fbeb9937","time_created":"2024-11-28T01:53:03.159475Z"}
23352024-11-28T02:02:21.323ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = c9d88c22-5228-416e-9e59-fb7d4baea927 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
23362024-11-28T02:02:21.323ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = c9d88c22-5228-416e-9e59-fb7d4baea927 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
23372024-11-28T02:02:21.325ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2329 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = c9d88c22-5228-416e-9e59-fb7d4baea927 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
23382024-11-28T02:02:22.344ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 9924608a-51a4-4fda-a7d5-137e2a95a576 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
23392024-11-28T02:02:22.344ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 9924608a-51a4-4fda-a7d5-137e2a95a576 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
23402024-11-28T02:02:22.346ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2262 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 9924608a-51a4-4fda-a7d5-137e2a95a576 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
23412024-11-28T02:02:23.365ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 28df4354-2280-4a06-8a03-45182342673f roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
23422024-11-28T02:02:23.365ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 28df4354-2280-4a06-8a03-45182342673f resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
23432024-11-28T02:02:23.367ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2492 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 28df4354-2280-4a06-8a03-45182342673f response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
23442024-11-28T02:02:24.186ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:55495 req_id = d303a438-0857-46c7-a2d2-d521553db459 roles = RoleSet { roles: {} } uri = /metrics/collectors/119eed64-2e24-4d5f-8c42-be6cd8bec8b5/producers?sort_by=id_ascending
23452024-11-28T02:02:24.186ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:55495 req_id = d303a438-0857-46c7-a2d2-d521553db459 resource = Database result = Ok(()) uri = /metrics/collectors/119eed64-2e24-4d5f-8c42-be6cd8bec8b5/producers?sort_by=id_ascending
23462024-11-28T02:02:24.188ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2932 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:55495 req_id = d303a438-0857-46c7-a2d2-d521553db459 response_code = 200 uri = /metrics/collectors/119eed64-2e24-4d5f-8c42-be6cd8bec8b5/producers?sort_by=id_ascending
23472024-11-28T02:02:24.189ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:55495 req_id = fd1c8deb-50c6-49ca-a225-789f87d9df22 roles = RoleSet { roles: {} } uri = /metrics/collectors/119eed64-2e24-4d5f-8c42-be6cd8bec8b5/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJjNDM0YjQ0OC01MTk5LTQ2NmUtYWYzMi00ZjYyOWUzMWM3YjEifX0%3D
23482024-11-28T02:02:24.189ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:55495 req_id = fd1c8deb-50c6-49ca-a225-789f87d9df22 resource = Database result = Ok(()) uri = /metrics/collectors/119eed64-2e24-4d5f-8c42-be6cd8bec8b5/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJjNDM0YjQ0OC01MTk5LTQ2NmUtYWYzMi00ZjYyOWUzMWM3YjEifX0%3D
23492024-11-28T02:02:24.191ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2152 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:55495 req_id = fd1c8deb-50c6-49ca-a225-789f87d9df22 response_code = 200 uri = /metrics/collectors/119eed64-2e24-4d5f-8c42-be6cd8bec8b5/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJjNDM0YjQ0OC01MTk5LTQ2NmUtYWYzMi00ZjYyOWUzMWM3YjEifX0%3D
23502024-11-28T02:02:24.194ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1228 local_addr = [fd00:1122:3344:101::b]:55621 method = GET remote_addr = [fd00:1122:3344:101::d]:56417 req_id = a4217e54-8403-471e-a385-8b5ba553cc7a response_code = 200 uri = /2469a748-bcd2-4e0d-99cc-f79def0452cc
23512024-11-28T02:02:24.387ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 390799ea-50fa-4b3e-9654-515e92fe763d roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
23522024-11-28T02:02:24.387ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 390799ea-50fa-4b3e-9654-515e92fe763d resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
23532024-11-28T02:02:24.389ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2128 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 390799ea-50fa-4b3e-9654-515e92fe763d response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
23542024-11-28T02:02:25.409ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 407d15ea-2113-446d-a702-f1e3e9a0ba79 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
23552024-11-28T02:02:25.409ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 407d15ea-2113-446d-a702-f1e3e9a0ba79 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
23562024-11-28T02:02:25.411ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2119 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 407d15ea-2113-446d-a702-f1e3e9a0ba79 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
23572024-11-28T02:02:26.430ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = e7b6747d-a322-4ca5-a0f6-2ea1d0d6aea3 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
23582024-11-28T02:02:26.430ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = e7b6747d-a322-4ca5-a0f6-2ea1d0d6aea3 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
23592024-11-28T02:02:26.432ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2244 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = e7b6747d-a322-4ca5-a0f6-2ea1d0d6aea3 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
23602024-11-28T02:02:27.451ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 74161476-9cb1-422c-aa1f-b6c6fa8340d1 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
23612024-11-28T02:02:27.451ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 74161476-9cb1-422c-aa1f-b6c6fa8340d1 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
23622024-11-28T02:02:27.453ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2500 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 74161476-9cb1-422c-aa1f-b6c6fa8340d1 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
23632024-11-28T02:02:28.472ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 2003a7e8-dcbe-4a5d-9111-f6b4378d86ca roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
23642024-11-28T02:02:28.473ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 2003a7e8-dcbe-4a5d-9111-f6b4378d86ca resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
23652024-11-28T02:02:28.475ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2861 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 2003a7e8-dcbe-4a5d-9111-f6b4378d86ca response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
23662024-11-28T02:02:29.494ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 770756a3-b021-41e6-8749-e54ef40b737d roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
23672024-11-28T02:02:29.494ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 770756a3-b021-41e6-8749-e54ef40b737d resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
23682024-11-28T02:02:29.496ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2276 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 770756a3-b021-41e6-8749-e54ef40b737d response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
23692024-11-28T02:02:30.478ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 57 reason = Timeout
23702024-11-28T02:02:30.478ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
23712024-11-28T02:02:30.478ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 57 reason = Timeout
23722024-11-28T02:02:30.478ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
23732024-11-28T02:02:30.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
23742024-11-28T02:02:30.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
23752024-11-28T02:02:30.480ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23762024-11-28T02:02:30.481ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
23772024-11-28T02:02:30.481ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
23782024-11-28T02:02:30.483ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
23792024-11-28T02:02:30.483ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
23802024-11-28T02:02:30.483ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
23812024-11-28T02:02:30.483ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 4.647671ms iteration = 57 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
23822024-11-28T02:02:30.484ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
23832024-11-28T02:02:30.484ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
23842024-11-28T02:02:30.486ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23852024-11-28T02:02:30.488ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 67ac247f-c934-4c78-857a-fba4fbeb9937, lookup_type: ById(67ac247f-c934-4c78-857a-fba4fbeb9937) } result = Ok(())
23862024-11-28T02:02:30.488ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
23872024-11-28T02:02:30.488ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
23882024-11-28T02:02:30.498ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 67ac247f-c934-4c78-857a-fba4fbeb9937 original_time_created = 2024-11-28 01:53:03.159475 UTC target_id = 67ac247f-c934-4c78-857a-fba4fbeb9937 time_created = 2024-11-28 01:53:03.159475 UTC
23892024-11-28T02:02:30.498ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 19.53238ms iteration = 57 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"67ac247f-c934-4c78-857a-fba4fbeb9937","time_created":"2024-11-28T01:53:03.159475Z"}
23902024-11-28T02:02:30.516ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 24a473a6-3400-4bcb-b303-c3e675428fdc roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
23912024-11-28T02:02:30.516ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 24a473a6-3400-4bcb-b303-c3e675428fdc resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
23922024-11-28T02:02:30.517ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1902 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 24a473a6-3400-4bcb-b303-c3e675428fdc response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
23932024-11-28T02:02:31.537ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 0dd0c979-1397-4610-ac84-182285791e70 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
23942024-11-28T02:02:31.537ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 0dd0c979-1397-4610-ac84-182285791e70 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
23952024-11-28T02:02:31.539ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2053 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 0dd0c979-1397-4610-ac84-182285791e70 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
23962024-11-28T02:02:32.558ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = c738d681-b684-4e08-8386-649e189c7adc roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
23972024-11-28T02:02:32.559ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = c738d681-b684-4e08-8386-649e189c7adc resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
23982024-11-28T02:02:32.561ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2318 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = c738d681-b684-4e08-8386-649e189c7adc response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
23992024-11-28T02:02:33.579ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 9c2c9e20-6010-472c-ab2a-36d56467abfc roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
24002024-11-28T02:02:33.580ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 9c2c9e20-6010-472c-ab2a-36d56467abfc resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
24012024-11-28T02:02:33.581ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2384 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 9c2c9e20-6010-472c-ab2a-36d56467abfc response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
24022024-11-28T02:02:34.194ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1194 local_addr = [fd00:1122:3344:101::b]:55621 method = GET remote_addr = [fd00:1122:3344:101::d]:56417 req_id = b16f9485-426a-4b8c-8efa-bdaf85265071 response_code = 200 uri = /2469a748-bcd2-4e0d-99cc-f79def0452cc
24032024-11-28T02:02:34.600ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = cb4f367e-96e2-4a04-983f-7570a12ef0a7 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
24042024-11-28T02:02:34.600ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = cb4f367e-96e2-4a04-983f-7570a12ef0a7 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
24052024-11-28T02:02:34.602ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2020 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = cb4f367e-96e2-4a04-983f-7570a12ef0a7 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
24062024-11-28T02:02:35.620ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 0ece627f-2ae5-4b76-8b00-b7c2582d80a0 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
24072024-11-28T02:02:35.621ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 0ece627f-2ae5-4b76-8b00-b7c2582d80a0 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
24082024-11-28T02:02:35.623ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2605 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 0ece627f-2ae5-4b76-8b00-b7c2582d80a0 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
24092024-11-28T02:02:36.641ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 04a3e6f5-7967-476b-92c0-f012b7b94c64 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
24102024-11-28T02:02:36.642ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 04a3e6f5-7967-476b-92c0-f012b7b94c64 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
24112024-11-28T02:02:36.643ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2247 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 04a3e6f5-7967-476b-92c0-f012b7b94c64 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
24122024-11-28T02:02:37.665ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 10c7398a-6719-4336-9125-e9994d1e2606 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
24132024-11-28T02:02:37.666ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 10c7398a-6719-4336-9125-e9994d1e2606 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
24142024-11-28T02:02:37.668ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2384 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 10c7398a-6719-4336-9125-e9994d1e2606 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
24152024-11-28T02:02:38.686ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = a056d7d2-c1fd-4649-97e6-82820e84c00c roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
24162024-11-28T02:02:38.687ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = a056d7d2-c1fd-4649-97e6-82820e84c00c resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
24172024-11-28T02:02:38.688ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2020 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = a056d7d2-c1fd-4649-97e6-82820e84c00c response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
24182024-11-28T02:02:39.707ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = ba47b6b1-4ad7-4ecc-a3f8-2241dc2ea37e roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
24192024-11-28T02:02:39.707ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = ba47b6b1-4ad7-4ecc-a3f8-2241dc2ea37e resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
24202024-11-28T02:02:39.709ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1915 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = ba47b6b1-4ad7-4ecc-a3f8-2241dc2ea37e response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
24212024-11-28T02:02:40.478ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 58 reason = Timeout
24222024-11-28T02:02:40.478ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
24232024-11-28T02:02:40.478ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 20 reason = Timeout
24242024-11-28T02:02:40.478ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
24252024-11-28T02:02:40.478ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 20 reason = Timeout
24262024-11-28T02:02:40.478ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
24272024-11-28T02:02:40.478ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 21 reason = Timeout
24282024-11-28T02:02:40.478ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
24292024-11-28T02:02:40.478ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 20 reason = Timeout
24302024-11-28T02:02:40.478ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
24312024-11-28T02:02:40.478ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
24322024-11-28T02:02:40.478ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 20 reason = Timeout
24332024-11-28T02:02:40.478ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 58 reason = Timeout
24342024-11-28T02:02:40.478ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 20 reason = Timeout
24352024-11-28T02:02:40.478ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
24362024-11-28T02:02:40.478ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 20 reason = Timeout
24372024-11-28T02:02:40.478ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 20 reason = Timeout
24382024-11-28T02:02:40.478ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
24392024-11-28T02:02:40.478ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
24402024-11-28T02:02:40.478ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
24412024-11-28T02:02:40.478ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
24422024-11-28T02:02:40.478ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 20 reason = Timeout
24432024-11-28T02:02:40.478ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
24442024-11-28T02:02:40.478ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 20 reason = Timeout
24452024-11-28T02:02:40.478ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
24462024-11-28T02:02:40.478ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 20 reason = Timeout
24472024-11-28T02:02:40.478ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
24482024-11-28T02:02:40.478ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 20 reason = Timeout
24492024-11-28T02:02:40.478ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
24502024-11-28T02:02:40.478ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 20 reason = Timeout
24512024-11-28T02:02:40.478ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
24522024-11-28T02:02:40.478ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 20 reason = Timeout
24532024-11-28T02:02:40.479ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
24542024-11-28T02:02:40.479ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 23 reason = Timeout
24552024-11-28T02:02:40.479ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
24562024-11-28T02:02:40.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
24572024-11-28T02:02:40.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
24582024-11-28T02:02:40.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
24592024-11-28T02:02:40.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
24602024-11-28T02:02:40.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
24612024-11-28T02:02:40.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
24622024-11-28T02:02:40.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
24632024-11-28T02:02:40.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
24642024-11-28T02:02:40.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
24652024-11-28T02:02:40.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
24662024-11-28T02:02:40.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
24672024-11-28T02:02:40.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
24682024-11-28T02:02:40.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
24692024-11-28T02:02:40.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
24702024-11-28T02:02:40.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
24712024-11-28T02:02:40.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
24722024-11-28T02:02:40.481ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
24732024-11-28T02:02:40.481ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 3.159005ms iteration = 20 status = {"errors":[],"garbage_collect_requested":[]}
24742024-11-28T02:02:40.482ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
24752024-11-28T02:02:40.482ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.729145ms iteration = 20 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
24762024-11-28T02:02:40.483ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
24772024-11-28T02:02:40.483ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
24782024-11-28T02:02:40.484ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
24792024-11-28T02:02:40.484ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
24802024-11-28T02:02:40.484ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
24812024-11-28T02:02:40.484ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
24822024-11-28T02:02:40.484ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
24832024-11-28T02:02:40.485ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
24842024-11-28T02:02:40.486ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
24852024-11-28T02:02:40.487ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-313847b3-d670-4167-9b09-9e7d3dfcb2a4.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-313847b3-d670-4167-9b09-9e7d3dfcb2a4.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1426, tv_nsec: 957425114 } })
24862024-11-28T02:02:40.487ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-313847b3-d670-4167-9b09-9e7d3dfcb2a4.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-313847b3-d670-4167-9b09-9e7d3dfcb2a4.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1426, tv_nsec: 957589147 } })
24872024-11-28T02:02:40.487ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1103
24882024-11-28T02:02:40.487ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1103
24892024-11-28T02:02:40.487ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
24902024-11-28T02:02:40.488ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
24912024-11-28T02:02:40.488ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
24922024-11-28T02:02:40.488ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
24932024-11-28T02:02:40.489ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
24942024-11-28T02:02:40.489ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
24952024-11-28T02:02:40.489ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
24962024-11-28T02:02:40.490ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
24972024-11-28T02:02:40.490ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
24982024-11-28T02:02:40.490ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
24992024-11-28T02:02:40.490ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
25002024-11-28T02:02:40.491ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25012024-11-28T02:02:40.493ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25022024-11-28T02:02:40.493ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 14.649535ms iteration = 58 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
25032024-11-28T02:02:40.493ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
25042024-11-28T02:02:40.493ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
25052024-11-28T02:02:40.493ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
25062024-11-28T02:02:40.493ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
25072024-11-28T02:02:40.494ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
25082024-11-28T02:02:40.495ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
25092024-11-28T02:02:40.495ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
25102024-11-28T02:02:40.495ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
25112024-11-28T02:02:40.495ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25122024-11-28T02:02:40.496ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-313847b3-d670-4167-9b09-9e7d3dfcb2a4.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-313847b3-d670-4167-9b09-9e7d3dfcb2a4.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1426, tv_nsec: 966905459 } })
25132024-11-28T02:02:40.496ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1103 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc
25142024-11-28T02:02:40.496ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
25152024-11-28T02:02:40.496ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
25162024-11-28T02:02:40.496ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
25172024-11-28T02:02:40.496ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
25182024-11-28T02:02:40.497ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
25192024-11-28T02:02:40.497ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
25202024-11-28T02:02:40.497ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
25212024-11-28T02:02:40.497ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
25222024-11-28T02:02:40.498ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
25232024-11-28T02:02:40.498ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
25242024-11-28T02:02:40.498ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
25252024-11-28T02:02:40.500ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
25262024-11-28T02:02:40.500ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
25272024-11-28T02:02:40.501ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25282024-11-28T02:02:40.501ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
25292024-11-28T02:02:40.501ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 22.855067ms iteration = 20 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
25302024-11-28T02:02:40.502ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25312024-11-28T02:02:40.505ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
25322024-11-28T02:02:40.505ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
25332024-11-28T02:02:40.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
25342024-11-28T02:02:40.506ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 67ac247f-c934-4c78-857a-fba4fbeb9937, lookup_type: ById(67ac247f-c934-4c78-857a-fba4fbeb9937) } result = Ok(())
25352024-11-28T02:02:40.506ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
25362024-11-28T02:02:40.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
25372024-11-28T02:02:40.523ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 45.128374ms iteration = 23 status = {"physical_disks_added":0}
25382024-11-28T02:02:40.524ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 67ac247f-c934-4c78-857a-fba4fbeb9937 original_time_created = 2024-11-28 01:53:03.159475 UTC target_id = 67ac247f-c934-4c78-857a-fba4fbeb9937 time_created = 2024-11-28 01:53:03.159475 UTC
25392024-11-28T02:02:40.525ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 46.312085ms iteration = 58 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"67ac247f-c934-4c78-857a-fba4fbeb9937","time_created":"2024-11-28T01:53:03.159475Z"}
25402024-11-28T02:02:40.530ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 52.219027ms iteration = 20 status = {"errors":[],"records_set_to_done":[]}
25412024-11-28T02:02:40.531ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
25422024-11-28T02:02:40.532ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
25432024-11-28T02:02:40.532ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
25442024-11-28T02:02:40.532ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
25452024-11-28T02:02:40.532ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
25462024-11-28T02:02:40.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
25472024-11-28T02:02:40.537ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
25482024-11-28T02:02:40.537ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
25492024-11-28T02:02:40.537ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 58.332123ms iteration = 20 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
25502024-11-28T02:02:40.537ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
25512024-11-28T02:02:40.537ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
25522024-11-28T02:02:40.597ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
25532024-11-28T02:02:40.597ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
25542024-11-28T02:02:40.597ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "ba89fa85-003e-4717-9d82-d154610b1ba4", "content-length": "26", "date": "Thu, 28 Nov 2024 02:02:40 GMT"} })
25552024-11-28T02:02:40.597ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
25562024-11-28T02:02:40.599ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
25572024-11-28T02:02:40.614ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
25582024-11-28T02:02:40.614ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
25592024-11-28T02:02:40.636ZINFOnexus (BackgroundTasks): VPC route manager sled 313847b3-d670-4167-9b09-9e7d3dfcb2a4 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
25602024-11-28T02:02:40.636ZDEBGnexus (BackgroundTasks): client request SledAgent = 313847b3-d670-4167-9b09-9e7d3dfcb2a4 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
25612024-11-28T02:02:40.636ZDEBGnexus (BackgroundTasks): client response SledAgent = 313847b3-d670-4167-9b09-9e7d3dfcb2a4 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "bbc36826-a2af-425e-847a-83d5ba65e43b", "content-length": "372", "date": "Thu, 28 Nov 2024 02:02:40 GMT"} })
25622024-11-28T02:02:40.636ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
25632024-11-28T02:02:40.637ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
25642024-11-28T02:02:40.638ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
25652024-11-28T02:02:40.639ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
25662024-11-28T02:02:40.639ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 160.761582ms iteration = 20 status = {"errors":[],"requests_created_ok":[],"start_invoked_ok":[]}
25672024-11-28T02:02:40.640ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
25682024-11-28T02:02:40.640ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
25692024-11-28T02:02:40.641ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25702024-11-28T02:02:40.641ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 162.531112ms iteration = 20 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
25712024-11-28T02:02:40.642ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 163.370968ms iteration = 20 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[]}
25722024-11-28T02:02:40.642ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25732024-11-28T02:02:40.643ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
25742024-11-28T02:02:40.643ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
25752024-11-28T02:02:40.643ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
25762024-11-28T02:02:40.645ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 313847b3-d670-4167-9b09-9e7d3dfcb2a4, lookup_type: ById(313847b3-d670-4167-9b09-9e7d3dfcb2a4) } result = Ok(())
25772024-11-28T02:02:40.645ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
25782024-11-28T02:02:40.645ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
25792024-11-28T02:02:40.647ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25802024-11-28T02:02:40.647ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 169.223099ms iteration = 20 status = 0
25812024-11-28T02:02:40.649ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
25822024-11-28T02:02:40.649ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
25832024-11-28T02:02:40.649ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
25842024-11-28T02:02:40.654ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
25852024-11-28T02:02:40.654ZDEBGnexus (BackgroundTasks): client request SledAgent = 313847b3-d670-4167-9b09-9e7d3dfcb2a4 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
25862024-11-28T02:02:40.654ZDEBGnexus (BackgroundTasks): client response SledAgent = 313847b3-d670-4167-9b09-9e7d3dfcb2a4 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "abb7c9a7-3a2d-41ba-baae-3d395d782d07", "date": "Thu, 28 Nov 2024 02:02:40 GMT"} })
25872024-11-28T02:02:40.654ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
25882024-11-28T02:02:40.655ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
25892024-11-28T02:02:40.661ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
25902024-11-28T02:02:40.661ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
25912024-11-28T02:02:40.665ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
25922024-11-28T02:02:40.665ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
25932024-11-28T02:02:40.668ZINFOnexus (BackgroundTasks): VPC route manager sled 313847b3-d670-4167-9b09-9e7d3dfcb2a4 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
25942024-11-28T02:02:40.668ZDEBGnexus (BackgroundTasks): client request SledAgent = 313847b3-d670-4167-9b09-9e7d3dfcb2a4 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
25952024-11-28T02:02:40.712ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1111 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc zone_address = fd00:1122:3344:101::2
25962024-11-28T02:02:40.712ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
25972024-11-28T02:02:40.712ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "6997505b-5787-4fa6-85e5-dff1fafaf427", "content-length": "26", "date": "Thu, 28 Nov 2024 02:02:40 GMT"} })
25982024-11-28T02:02:40.712ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1116 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
25992024-11-28T02:02:40.715ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc
26002024-11-28T02:02:40.716ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
26012024-11-28T02:02:40.716ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
26022024-11-28T02:02:40.716ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 237.535056ms iteration = 20 status = {}
26032024-11-28T02:02:40.716ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "a5067bf9-fa36-42f2-adc0-c72ce2babfb7", "content-length": "26", "date": "Thu, 28 Nov 2024 02:02:40 GMT"} })
26042024-11-28T02:02:40.716ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
26052024-11-28T02:02:40.718ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
26062024-11-28T02:02:40.785ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 9ca42632-8a0e-4f28-b132-3757a002f253 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
26072024-11-28T02:02:40.785ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 9ca42632-8a0e-4f28-b132-3757a002f253 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
26082024-11-28T02:02:40.785ZDEBGnexus (BackgroundTasks): client response SledAgent = 313847b3-d670-4167-9b09-9e7d3dfcb2a4 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "24e3607c-b4d0-458f-88e1-bd1dba0609df", "date": "Thu, 28 Nov 2024 02:02:40 GMT"} })
26092024-11-28T02:02:40.788ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2866 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 9ca42632-8a0e-4f28-b132-3757a002f253 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
26102024-11-28T02:02:40.788ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 310.153448ms iteration = 21 status = {}
26112024-11-28T02:02:40.790ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
26122024-11-28T02:02:40.791ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "e116a5d1-3a9b-49a6-87b0-6cb064caa216", "content-length": "1", "date": "Thu, 28 Nov 2024 02:02:40 GMT"} })
26132024-11-28T02:02:40.792ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
26142024-11-28T02:02:40.792ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
26152024-11-28T02:02:40.794ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 315.867576ms iteration = 20 status = null
26162024-11-28T02:02:40.796ZDEBGnexus (BackgroundTasks): client request SledAgent = 313847b3-d670-4167-9b09-9e7d3dfcb2a4 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
26172024-11-28T02:02:40.796ZDEBGnexus (BackgroundTasks): client response SledAgent = 313847b3-d670-4167-9b09-9e7d3dfcb2a4 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "42695025-fdf4-4b99-8ca9-e2486b686601", "content-length": "691", "date": "Thu, 28 Nov 2024 02:02:40 GMT"} })
26182024-11-28T02:02:40.797ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
26192024-11-28T02:02:40.797ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
26202024-11-28T02:02:40.797ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
26212024-11-28T02:02:40.798ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 320.213462ms iteration = 20 status = {}
26222024-11-28T02:02:40.850ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
26232024-11-28T02:02:40.850ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
26242024-11-28T02:02:40.853ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 4bb2986f-2e92-4ac3-9dc1-591b6bb60b3e, rack_id: 92a8ff6e-21d3-4c28-bcad-e323847985dc, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(ea300318-9d62-42ed-80ea-21c86590a166) } rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc switch_location = Switch0
26252024-11-28T02:02:40.853ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
26262024-11-28T02:02:40.853ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
26272024-11-28T02:02:40.862ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
26282024-11-28T02:02:40.863ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "d111d929-c83b-43b1-b0a3-54a19369e215", "content-length": "134", "date": "Thu, 28 Nov 2024 02:02:40 GMT"} })
26292024-11-28T02:02:40.863ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1746 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
26302024-11-28T02:02:40.863ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1773 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = ea300318-9d62-42ed-80ea-21c86590a166
26312024-11-28T02:02:40.863ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
26322024-11-28T02:02:40.863ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
26332024-11-28T02:02:40.863ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
26342024-11-28T02:02:40.863ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
26352024-11-28T02:02:40.863ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
26362024-11-28T02:02:40.863ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1920 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
26372024-11-28T02:02:40.863ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
26382024-11-28T02:02:40.863ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
26392024-11-28T02:02:40.864ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc
26402024-11-28T02:02:40.864ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
26412024-11-28T02:02:40.864ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
26422024-11-28T02:02:40.866ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
26432024-11-28T02:02:40.866ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "6433d8c9-d840-4148-9826-5124553d4f33", "content-length": "2", "date": "Thu, 28 Nov 2024 02:02:40 GMT"} })
26442024-11-28T02:02:40.866ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
26452024-11-28T02:02:40.866ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "3ce1546b-16c9-49a1-97ce-1cd3a2231af0", "content-length": "52", "date": "Thu, 28 Nov 2024 02:02:40 GMT"} })
26462024-11-28T02:02:40.866ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc switch_location = Switch0
26472024-11-28T02:02:40.866ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
26482024-11-28T02:02:40.957ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "a169ea7e-c833-4ec8-96ab-5f4f8680f8d1", "date": "Thu, 28 Nov 2024 02:02:40 GMT"} })
26492024-11-28T02:02:40.957ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc switch = Switch0
26502024-11-28T02:02:40.957ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
26512024-11-28T02:02:40.957ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "8377e7d8-0382-4570-a750-86234405d576", "content-length": "682", "date": "Thu, 28 Nov 2024 02:02:40 GMT"} })
26522024-11-28T02:02:40.957ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
26532024-11-28T02:02:40.958ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
26542024-11-28T02:02:40.960ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
26552024-11-28T02:02:40.960ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
26562024-11-28T02:02:40.964ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
26572024-11-28T02:02:40.964ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
26582024-11-28T02:02:40.966ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
26592024-11-28T02:02:40.966ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
26602024-11-28T02:02:40.973ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 494.663862ms iteration = 20 status = {}
26612024-11-28T02:02:41.311ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:40402 req_id = ba4cff3e-d7df-49ad-9e6f-01d63334c3b6 roles = RoleSet { roles: {} } uri = /nat/ipv4/changeset/6?limit=100
26622024-11-28T02:02:41.312ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:40402 req_id = ba4cff3e-d7df-49ad-9e6f-01d63334c3b6 resource = Database result = Ok(()) uri = /nat/ipv4/changeset/6?limit=100
26632024-11-28T02:02:41.314ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2886 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:40402 req_id = ba4cff3e-d7df-49ad-9e6f-01d63334c3b6 response_code = 200 uri = /nat/ipv4/changeset/6?limit=100
26642024-11-28T02:02:41.808ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 732b5f89-a568-460d-9600-61106bb5fbda roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
26652024-11-28T02:02:41.809ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 732b5f89-a568-460d-9600-61106bb5fbda resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
26662024-11-28T02:02:41.811ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2464 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 732b5f89-a568-460d-9600-61106bb5fbda response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
26672024-11-28T02:02:42.832ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 7ba7727a-3802-4c80-b82e-bba44c5e754d roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
26682024-11-28T02:02:42.832ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 7ba7727a-3802-4c80-b82e-bba44c5e754d resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
26692024-11-28T02:02:42.835ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2491 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 7ba7727a-3802-4c80-b82e-bba44c5e754d response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
26702024-11-28T02:02:43.853ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 146c9da4-2604-453c-9c92-c25b94101eee roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
26712024-11-28T02:02:43.854ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 146c9da4-2604-453c-9c92-c25b94101eee resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
26722024-11-28T02:02:43.856ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2362 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 146c9da4-2604-453c-9c92-c25b94101eee response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
26732024-11-28T02:02:44.194ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1631 local_addr = [fd00:1122:3344:101::b]:55621 method = GET remote_addr = [fd00:1122:3344:101::d]:56417 req_id = f0987ce8-2b1e-43b4-a4ff-652b9874e765 response_code = 200 uri = /2469a748-bcd2-4e0d-99cc-f79def0452cc
26742024-11-28T02:02:44.875ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 1c5b6535-f01e-452e-8133-0a8f053d9f5b roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
26752024-11-28T02:02:44.875ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 1c5b6535-f01e-452e-8133-0a8f053d9f5b resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
26762024-11-28T02:02:44.878ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2693 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 1c5b6535-f01e-452e-8133-0a8f053d9f5b response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
26772024-11-28T02:02:45.895ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = fdbe2b35-3dbd-4205-8e33-c4ca260fa1e2 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
26782024-11-28T02:02:45.896ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = fdbe2b35-3dbd-4205-8e33-c4ca260fa1e2 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
26792024-11-28T02:02:45.897ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2159 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = fdbe2b35-3dbd-4205-8e33-c4ca260fa1e2 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
26802024-11-28T02:02:46.917ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = e87f99c8-b2cd-4a3b-9926-42efbb8c437f roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
26812024-11-28T02:02:46.917ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = e87f99c8-b2cd-4a3b-9926-42efbb8c437f resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
26822024-11-28T02:02:46.919ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2034 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = e87f99c8-b2cd-4a3b-9926-42efbb8c437f response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
26832024-11-28T02:02:47.938ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = c82f67a0-e373-4256-a87d-2026f8f5a445 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
26842024-11-28T02:02:47.938ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = c82f67a0-e373-4256-a87d-2026f8f5a445 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
26852024-11-28T02:02:47.940ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2038 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = c82f67a0-e373-4256-a87d-2026f8f5a445 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
26862024-11-28T02:02:48.958ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 654f8309-8f5a-43de-9adb-4a5a46004373 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
26872024-11-28T02:02:48.958ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 654f8309-8f5a-43de-9adb-4a5a46004373 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
26882024-11-28T02:02:48.960ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2068 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 654f8309-8f5a-43de-9adb-4a5a46004373 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
26892024-11-28T02:02:49.978ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 01d7386d-1b18-4457-a255-adcadcbc239d roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
26902024-11-28T02:02:49.979ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 01d7386d-1b18-4457-a255-adcadcbc239d resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
26912024-11-28T02:02:49.980ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2155 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 01d7386d-1b18-4457-a255-adcadcbc239d response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
26922024-11-28T02:02:50.478ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 59 reason = Timeout
26932024-11-28T02:02:50.478ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 59 reason = Timeout
26942024-11-28T02:02:50.478ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
26952024-11-28T02:02:50.478ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
26962024-11-28T02:02:50.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
26972024-11-28T02:02:50.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
26982024-11-28T02:02:50.481ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26992024-11-28T02:02:50.484ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
27002024-11-28T02:02:50.484ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
27012024-11-28T02:02:50.484ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
27022024-11-28T02:02:50.486ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
27032024-11-28T02:02:50.486ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
27042024-11-28T02:02:50.488ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
27052024-11-28T02:02:50.490ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 67ac247f-c934-4c78-857a-fba4fbeb9937, lookup_type: ById(67ac247f-c934-4c78-857a-fba4fbeb9937) } result = Ok(())
27062024-11-28T02:02:50.490ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
27072024-11-28T02:02:50.490ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
27082024-11-28T02:02:50.499ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 67ac247f-c934-4c78-857a-fba4fbeb9937 original_time_created = 2024-11-28 01:53:03.159475 UTC target_id = 67ac247f-c934-4c78-857a-fba4fbeb9937 time_created = 2024-11-28 01:53:03.159475 UTC
27092024-11-28T02:02:50.499ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 20.579428ms iteration = 59 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"67ac247f-c934-4c78-857a-fba4fbeb9937","time_created":"2024-11-28T01:53:03.159475Z"}
27102024-11-28T02:02:50.507ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
27112024-11-28T02:02:50.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
27122024-11-28T02:02:50.534ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 55.174298ms iteration = 59 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
27132024-11-28T02:02:50.999ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 32ccc3fb-721c-44d5-9504-105c69b55917 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
27142024-11-28T02:02:51.000ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 32ccc3fb-721c-44d5-9504-105c69b55917 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
27152024-11-28T02:02:51.001ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2109 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 32ccc3fb-721c-44d5-9504-105c69b55917 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
27162024-11-28T02:02:52.021ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = ce80a796-271e-4c2a-a0c4-383e54d36b8a roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
27172024-11-28T02:02:52.022ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = ce80a796-271e-4c2a-a0c4-383e54d36b8a resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
27182024-11-28T02:02:52.023ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2253 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = ce80a796-271e-4c2a-a0c4-383e54d36b8a response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
27192024-11-28T02:02:53.042ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 8653f768-a14c-4037-b41e-af1b70ad1a58 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
27202024-11-28T02:02:53.043ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 8653f768-a14c-4037-b41e-af1b70ad1a58 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
27212024-11-28T02:02:53.045ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2421 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 8653f768-a14c-4037-b41e-af1b70ad1a58 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
27222024-11-28T02:02:54.063ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = abb3655b-f8fe-4c76-9d19-3221a5fc6935 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
27232024-11-28T02:02:54.064ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = abb3655b-f8fe-4c76-9d19-3221a5fc6935 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
27242024-11-28T02:02:54.066ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2817 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = abb3655b-f8fe-4c76-9d19-3221a5fc6935 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
27252024-11-28T02:02:54.186ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:55495 req_id = 7e988b6f-c81f-4b5b-9502-70454e935549 roles = RoleSet { roles: {} } uri = /metrics/collectors/119eed64-2e24-4d5f-8c42-be6cd8bec8b5/producers?sort_by=id_ascending
27262024-11-28T02:02:54.186ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:55495 req_id = 7e988b6f-c81f-4b5b-9502-70454e935549 resource = Database result = Ok(()) uri = /metrics/collectors/119eed64-2e24-4d5f-8c42-be6cd8bec8b5/producers?sort_by=id_ascending
27272024-11-28T02:02:54.188ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2723 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:55495 req_id = 7e988b6f-c81f-4b5b-9502-70454e935549 response_code = 200 uri = /metrics/collectors/119eed64-2e24-4d5f-8c42-be6cd8bec8b5/producers?sort_by=id_ascending
27282024-11-28T02:02:54.189ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:55495 req_id = a1ac07fd-4ae5-4fe5-b9c4-8996b3404aaf roles = RoleSet { roles: {} } uri = /metrics/collectors/119eed64-2e24-4d5f-8c42-be6cd8bec8b5/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJjNDM0YjQ0OC01MTk5LTQ2NmUtYWYzMi00ZjYyOWUzMWM3YjEifX0%3D
27292024-11-28T02:02:54.189ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:55495 req_id = a1ac07fd-4ae5-4fe5-b9c4-8996b3404aaf resource = Database result = Ok(()) uri = /metrics/collectors/119eed64-2e24-4d5f-8c42-be6cd8bec8b5/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJjNDM0YjQ0OC01MTk5LTQ2NmUtYWYzMi00ZjYyOWUzMWM3YjEifX0%3D
27302024-11-28T02:02:54.191ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2220 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:55495 req_id = a1ac07fd-4ae5-4fe5-b9c4-8996b3404aaf response_code = 200 uri = /metrics/collectors/119eed64-2e24-4d5f-8c42-be6cd8bec8b5/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJjNDM0YjQ0OC01MTk5LTQ2NmUtYWYzMi00ZjYyOWUzMWM3YjEifX0%3D
27312024-11-28T02:02:54.194ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1184 local_addr = [fd00:1122:3344:101::b]:55621 method = GET remote_addr = [fd00:1122:3344:101::d]:56417 req_id = d1ea4720-2dbd-4ffa-8e43-a13a6c792efc response_code = 200 uri = /2469a748-bcd2-4e0d-99cc-f79def0452cc
27322024-11-28T02:02:55.084ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 396bca0f-4581-4075-ad28-5506cb2c14e6 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
27332024-11-28T02:02:55.085ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 396bca0f-4581-4075-ad28-5506cb2c14e6 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
27342024-11-28T02:02:55.086ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1813 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 396bca0f-4581-4075-ad28-5506cb2c14e6 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
27352024-11-28T02:02:56.106ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 58c0a606-f3be-4b94-9487-ad5585e34f70 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
27362024-11-28T02:02:56.106ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 58c0a606-f3be-4b94-9487-ad5585e34f70 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
27372024-11-28T02:02:56.108ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2336 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 58c0a606-f3be-4b94-9487-ad5585e34f70 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
27382024-11-28T02:02:57.127ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 5c338688-0362-4414-be03-44fa92e0b6ef roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
27392024-11-28T02:02:57.127ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 5c338688-0362-4414-be03-44fa92e0b6ef resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
27402024-11-28T02:02:57.129ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2354 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 5c338688-0362-4414-be03-44fa92e0b6ef response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
27412024-11-28T02:02:58.148ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = ef2b3d62-5d00-4c37-ac84-84005c9f6146 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
27422024-11-28T02:02:58.148ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = ef2b3d62-5d00-4c37-ac84-84005c9f6146 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
27432024-11-28T02:02:58.150ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1784 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = ef2b3d62-5d00-4c37-ac84-84005c9f6146 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
27442024-11-28T02:02:59.169ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 48157abc-5ff8-4b63-938c-eaf454de1c52 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
27452024-11-28T02:02:59.169ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 48157abc-5ff8-4b63-938c-eaf454de1c52 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
27462024-11-28T02:02:59.171ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2217 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 48157abc-5ff8-4b63-938c-eaf454de1c52 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
27472024-11-28T02:03:00.190ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 98ab4905-4f78-43de-825e-d59c65ea078a roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
27482024-11-28T02:03:00.190ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 98ab4905-4f78-43de-825e-d59c65ea078a resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
27492024-11-28T02:03:00.192ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2495 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 98ab4905-4f78-43de-825e-d59c65ea078a response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
27502024-11-28T02:03:00.478ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 60 reason = Timeout
27512024-11-28T02:03:00.478ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 60 reason = Timeout
27522024-11-28T02:03:00.478ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
27532024-11-28T02:03:00.478ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
27542024-11-28T02:03:00.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
27552024-11-28T02:03:00.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
27562024-11-28T02:03:00.481ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
27572024-11-28T02:03:00.484ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
27582024-11-28T02:03:00.484ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
27592024-11-28T02:03:00.484ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
27602024-11-28T02:03:00.486ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
27612024-11-28T02:03:00.487ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
27622024-11-28T02:03:00.488ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
27632024-11-28T02:03:00.491ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 67ac247f-c934-4c78-857a-fba4fbeb9937, lookup_type: ById(67ac247f-c934-4c78-857a-fba4fbeb9937) } result = Ok(())
27642024-11-28T02:03:00.491ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
27652024-11-28T02:03:00.491ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
27662024-11-28T02:03:00.500ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 67ac247f-c934-4c78-857a-fba4fbeb9937 original_time_created = 2024-11-28 01:53:03.159475 UTC target_id = 67ac247f-c934-4c78-857a-fba4fbeb9937 time_created = 2024-11-28 01:53:03.159475 UTC
27672024-11-28T02:03:00.500ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 21.249919ms iteration = 60 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"67ac247f-c934-4c78-857a-fba4fbeb9937","time_created":"2024-11-28T01:53:03.159475Z"}
27682024-11-28T02:03:00.507ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
27692024-11-28T02:03:00.508ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
27702024-11-28T02:03:00.542ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 63.203588ms iteration = 60 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
27712024-11-28T02:03:01.211ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = bc175429-46c9-4e32-8b02-28d4f976729e roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
27722024-11-28T02:03:01.211ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = bc175429-46c9-4e32-8b02-28d4f976729e resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
27732024-11-28T02:03:01.213ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2382 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = bc175429-46c9-4e32-8b02-28d4f976729e response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
27742024-11-28T02:03:02.230ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 3de0e85f-a2f2-40bc-a15e-651eb88b92a8 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
27752024-11-28T02:03:02.231ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 3de0e85f-a2f2-40bc-a15e-651eb88b92a8 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
27762024-11-28T02:03:02.233ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2474 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 3de0e85f-a2f2-40bc-a15e-651eb88b92a8 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
27772024-11-28T02:03:03.252ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 5a8348b1-4822-4b6d-b6d1-66106ccf715d roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
27782024-11-28T02:03:03.252ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 5a8348b1-4822-4b6d-b6d1-66106ccf715d resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
27792024-11-28T02:03:03.254ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2075 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 5a8348b1-4822-4b6d-b6d1-66106ccf715d response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
27802024-11-28T02:03:04.195ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1501 local_addr = [fd00:1122:3344:101::b]:55621 method = GET remote_addr = [fd00:1122:3344:101::d]:56417 req_id = 798520d4-222d-40a0-805f-42a45b0a7682 response_code = 200 uri = /2469a748-bcd2-4e0d-99cc-f79def0452cc
27812024-11-28T02:03:04.274ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = f92ec53b-ca9c-4c9f-86bf-96839ec11e31 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
27822024-11-28T02:03:04.274ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = f92ec53b-ca9c-4c9f-86bf-96839ec11e31 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
27832024-11-28T02:03:04.276ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2474 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = f92ec53b-ca9c-4c9f-86bf-96839ec11e31 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
27842024-11-28T02:03:05.295ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 3bfc43ed-c130-4663-879d-88cad7b7e95a roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
27852024-11-28T02:03:05.295ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 3bfc43ed-c130-4663-879d-88cad7b7e95a resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
27862024-11-28T02:03:05.297ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2005 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 3bfc43ed-c130-4663-879d-88cad7b7e95a response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
27872024-11-28T02:03:06.317ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 799da012-07e3-41dc-8b74-48d64595028b roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
27882024-11-28T02:03:06.317ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 799da012-07e3-41dc-8b74-48d64595028b resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
27892024-11-28T02:03:06.318ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1808 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 799da012-07e3-41dc-8b74-48d64595028b response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
27902024-11-28T02:03:07.338ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 6839705c-da53-4ad7-a32e-253480d1fcc5 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
27912024-11-28T02:03:07.338ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 6839705c-da53-4ad7-a32e-253480d1fcc5 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
27922024-11-28T02:03:07.340ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1862 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 6839705c-da53-4ad7-a32e-253480d1fcc5 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
27932024-11-28T02:03:08.358ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 94a773ff-825a-4cab-8704-9f4662c1f09a roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
27942024-11-28T02:03:08.359ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 94a773ff-825a-4cab-8704-9f4662c1f09a resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
27952024-11-28T02:03:08.360ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2109 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 94a773ff-825a-4cab-8704-9f4662c1f09a response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
27962024-11-28T02:03:09.185ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:55495 req_id = b2dc1093-05ed-4d8a-ad03-bd384de70de1 roles = RoleSet { roles: {} } uri = /metrics/collectors/119eed64-2e24-4d5f-8c42-be6cd8bec8b5/producers?sort_by=id_ascending
27972024-11-28T02:03:09.186ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:55495 req_id = b2dc1093-05ed-4d8a-ad03-bd384de70de1 resource = Database result = Ok(()) uri = /metrics/collectors/119eed64-2e24-4d5f-8c42-be6cd8bec8b5/producers?sort_by=id_ascending
27982024-11-28T02:03:09.189ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 3530 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:55495 req_id = b2dc1093-05ed-4d8a-ad03-bd384de70de1 response_code = 200 uri = /metrics/collectors/119eed64-2e24-4d5f-8c42-be6cd8bec8b5/producers?sort_by=id_ascending
27992024-11-28T02:03:09.189ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:55495 req_id = 9b0cb374-ca6a-4122-b4b8-3fa1ad8acccb roles = RoleSet { roles: {} } uri = /metrics/collectors/119eed64-2e24-4d5f-8c42-be6cd8bec8b5/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJjNDM0YjQ0OC01MTk5LTQ2NmUtYWYzMi00ZjYyOWUzMWM3YjEifX0%3D
28002024-11-28T02:03:09.189ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:55495 req_id = 9b0cb374-ca6a-4122-b4b8-3fa1ad8acccb resource = Database result = Ok(()) uri = /metrics/collectors/119eed64-2e24-4d5f-8c42-be6cd8bec8b5/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJjNDM0YjQ0OC01MTk5LTQ2NmUtYWYzMi00ZjYyOWUzMWM3YjEifX0%3D
28012024-11-28T02:03:09.191ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2349 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:55495 req_id = 9b0cb374-ca6a-4122-b4b8-3fa1ad8acccb response_code = 200 uri = /metrics/collectors/119eed64-2e24-4d5f-8c42-be6cd8bec8b5/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJjNDM0YjQ0OC01MTk5LTQ2NmUtYWYzMi00ZjYyOWUzMWM3YjEifX0%3D
28022024-11-28T02:03:09.380ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 9c6ad968-040d-485c-8592-b9f501552823 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
28032024-11-28T02:03:09.380ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 9c6ad968-040d-485c-8592-b9f501552823 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
28042024-11-28T02:03:09.382ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2433 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 9c6ad968-040d-485c-8592-b9f501552823 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
28052024-11-28T02:03:09.911ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::b]:12221 remote_addr = [fd00:1122:3344:101::1]:34824
28062024-11-28T02:03:09.911ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::1]:34824 req_id = 0f478d20-a920-48a2-a721-08155ae9090d roles = RoleSet { roles: {} } uri = /metrics/producers
28072024-11-28T02:03:09.911ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::1]:34824 req_id = 0f478d20-a920-48a2-a721-08155ae9090d resource = Database result = Ok(()) uri = /metrics/producers
28082024-11-28T02:03:09.947ZINFOnexus: assigned collector to new producer collector_id = 119eed64-2e24-4d5f-8c42-be6cd8bec8b5 file = nexus/src/app/oximeter.rs:76 producer_id = 313847b3-d670-4167-9b09-9e7d3dfcb2a4
28092024-11-28T02:03:09.947ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 35793 local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::1]:34824 req_id = 0f478d20-a920-48a2-a721-08155ae9090d response_code = 201 uri = /metrics/producers
28102024-11-28T02:03:10.400ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 930c8fb8-9dec-4db6-8d93-9066fb0a559c roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
28112024-11-28T02:03:10.401ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 930c8fb8-9dec-4db6-8d93-9066fb0a559c resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
28122024-11-28T02:03:10.413ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 12462 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 930c8fb8-9dec-4db6-8d93-9066fb0a559c response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
28132024-11-28T02:03:10.478ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 13 reason = Timeout
28142024-11-28T02:03:10.478ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 21 reason = Timeout
28152024-11-28T02:03:10.478ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 12 reason = Timeout
28162024-11-28T02:03:10.478ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 21 reason = Timeout
28172024-11-28T02:03:10.478ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
28182024-11-28T02:03:10.478ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
28192024-11-28T02:03:10.478ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
28202024-11-28T02:03:10.478ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 21 reason = Timeout
28212024-11-28T02:03:10.478ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
28222024-11-28T02:03:10.478ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 21 reason = Timeout
28232024-11-28T02:03:10.478ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 61 reason = Timeout
28242024-11-28T02:03:10.478ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
28252024-11-28T02:03:10.478ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
28262024-11-28T02:03:10.478ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 12 reason = Timeout
28272024-11-28T02:03:10.478ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 21 reason = Timeout
28282024-11-28T02:03:10.478ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
28292024-11-28T02:03:10.478ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
28302024-11-28T02:03:10.478ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
28312024-11-28T02:03:10.478ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 61 reason = Timeout
28322024-11-28T02:03:10.478ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 21 reason = Timeout
28332024-11-28T02:03:10.478ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
28342024-11-28T02:03:10.478ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
28352024-11-28T02:03:10.478ZDEBGnexus (BackgroundTasks): activating background_task = inventory_collection iteration = 4 reason = Timeout
28362024-11-28T02:03:10.478ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
28372024-11-28T02:03:10.478ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 12 reason = Timeout
28382024-11-28T02:03:10.478ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 21 reason = Timeout
28392024-11-28T02:03:10.478ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
28402024-11-28T02:03:10.478ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 21 reason = Timeout
28412024-11-28T02:03:10.478ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
28422024-11-28T02:03:10.478ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 11 reason = Timeout
28432024-11-28T02:03:10.478ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:56
28442024-11-28T02:03:10.478ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
28452024-11-28T02:03:10.478ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 12 reason = Timeout
28462024-11-28T02:03:10.478ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
28472024-11-28T02:03:10.478ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
28482024-11-28T02:03:10.478ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 11 reason = Timeout
28492024-11-28T02:03:10.478ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
28502024-11-28T02:03:10.478ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 21 reason = Timeout
28512024-11-28T02:03:10.479ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
28522024-11-28T02:03:10.479ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 24 reason = Timeout
28532024-11-28T02:03:10.479ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
28542024-11-28T02:03:10.479ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 21 reason = Timeout
28552024-11-28T02:03:10.479ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
28562024-11-28T02:03:10.479ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 12 reason = Timeout
28572024-11-28T02:03:10.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
28582024-11-28T02:03:10.479ZDEBGnexus (BackgroundTasks): activating background_task = service_firewall_rule_propagation iteration = 3 reason = Timeout
28592024-11-28T02:03:10.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
28602024-11-28T02:03:10.479ZDEBGnexus (service-firewall-rule-progator): starting background task for service firewall rule propagation background_task = service_firewall_rule_propagation
28612024-11-28T02:03:10.479ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
28622024-11-28T02:03:10.479ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 21 reason = Timeout
28632024-11-28T02:03:10.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
28642024-11-28T02:03:10.479ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
28652024-11-28T02:03:10.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
28662024-11-28T02:03:10.479ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 21 reason = Timeout
28672024-11-28T02:03:10.479ZDEBGnexus (BackgroundTasks): activating background_task = saga_recovery iteration = 2 reason = Timeout
28682024-11-28T02:03:10.479ZDEBGnexus (SagaRecoverer): listing candidate sagas for recovery
28692024-11-28T02:03:10.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
28702024-11-28T02:03:10.479ZDEBGnexus (SagaRecoverer): roles roles = RoleSet { roles: {} }
28712024-11-28T02:03:10.479ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
28722024-11-28T02:03:10.479ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 11 reason = Timeout
28732024-11-28T02:03:10.479ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 22 reason = Timeout
28742024-11-28T02:03:10.479ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
28752024-11-28T02:03:10.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
28762024-11-28T02:03:10.479ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
28772024-11-28T02:03:10.479ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
28782024-11-28T02:03:10.479ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 13 reason = Timeout
28792024-11-28T02:03:10.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
28802024-11-28T02:03:10.479ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 12 reason = Timeout
28812024-11-28T02:03:10.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
28822024-11-28T02:03:10.479ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:83 reason = blueprint disabled target_id = 67ac247f-c934-4c78-857a-fba4fbeb9937
28832024-11-28T02:03:10.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
28842024-11-28T02:03:10.479ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 10.9\u{b5}s iteration = 12 status = {"enabled":false,"target_id":"67ac247f-c934-4c78-857a-fba4fbeb9937"}
28852024-11-28T02:03:10.479ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 21 reason = Timeout
28862024-11-28T02:03:10.479ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 11 reason = Timeout
28872024-11-28T02:03:10.479ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 11 reason = Timeout
28882024-11-28T02:03:10.479ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2024-11-28 01:53:10.479231029 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
28892024-11-28T02:03:10.479ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
28902024-11-28T02:03:10.479ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
28912024-11-28T02:03:10.479ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
28922024-11-28T02:03:10.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
28932024-11-28T02:03:10.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
28942024-11-28T02:03:10.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
28952024-11-28T02:03:10.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
28962024-11-28T02:03:10.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
28972024-11-28T02:03:10.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
28982024-11-28T02:03:10.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
28992024-11-28T02:03:10.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
29002024-11-28T02:03:10.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
29012024-11-28T02:03:10.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
29022024-11-28T02:03:10.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
29032024-11-28T02:03:10.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
29042024-11-28T02:03:10.479ZDEBGnexus (SagaRecoverer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000003a8a, .. }) resource = Database result = Ok(())
29052024-11-28T02:03:10.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
29062024-11-28T02:03:10.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
29072024-11-28T02:03:10.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
29082024-11-28T02:03:10.479ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("36d4f615-7461-4364-960c-89ece3aaabaf.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("eb274350-c0b0-4bce-9c8a-502880f2c905.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("eb274350-c0b0-4bce-9c8a-502880f2c905.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::8))) }], valid_until: Instant { tv_sec: 1456, tv_nsec: 949983115 } })
29092024-11-28T02:03:10.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
29102024-11-28T02:03:10.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
29112024-11-28T02:03:10.479ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("41ea4a99-3038-474e-bf28-2ad338f3243f.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("9402a51b-d30f-4544-93ec-0f5dc7a485e4.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("d23bc5f2-6b92-4992-bdd2-8695e5bf9c0f.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("d23bc5f2-6b92-4992-bdd2-8695e5bf9c0f.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:1::1))) }], valid_until: Instant { tv_sec: 1456, tv_nsec: 950227459 } })
29122024-11-28T02:03:10.480ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } dns_group = external
29132024-11-28T02:03:10.480ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.597288ms iteration = 12 status = {"addresses":["[fd00:1122:3344:101::9]:5353","[fd00:1122:3344:101::8]:5353"]}
29142024-11-28T02:03:10.480ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
29152024-11-28T02:03:10.480ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } dns_group = internal
29162024-11-28T02:03:10.480ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.448196ms iteration = 12 status = {"addresses":["[fd00:1122:3344:3::1]:5353","[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:1::1]:5353"]}
29172024-11-28T02:03:10.480ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
29182024-11-28T02:03:10.483ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
29192024-11-28T02:03:10.483ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
29202024-11-28T02:03:10.486ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 7.978699ms iteration = 21 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
29212024-11-28T02:03:10.489ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29222024-11-28T02:03:10.491ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29232024-11-28T02:03:10.493ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
29242024-11-28T02:03:10.493ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
29252024-11-28T02:03:10.493ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
29262024-11-28T02:03:10.495ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
29272024-11-28T02:03:10.495ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
29282024-11-28T02:03:10.496ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
29292024-11-28T02:03:10.499ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-313847b3-d670-4167-9b09-9e7d3dfcb2a4.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-313847b3-d670-4167-9b09-9e7d3dfcb2a4.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1456, tv_nsec: 969908921 } })
29302024-11-28T02:03:10.499ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29312024-11-28T02:03:10.499ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1103
29322024-11-28T02:03:10.502ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
29332024-11-28T02:03:10.502ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
29342024-11-28T02:03:10.503ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
29352024-11-28T02:03:10.503ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
29362024-11-28T02:03:10.503ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
29372024-11-28T02:03:10.506ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-313847b3-d670-4167-9b09-9e7d3dfcb2a4.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-313847b3-d670-4167-9b09-9e7d3dfcb2a4.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1456, tv_nsec: 976778671 } })
29382024-11-28T02:03:10.506ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
29392024-11-28T02:03:10.506ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1103
29402024-11-28T02:03:10.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
29412024-11-28T02:03:10.520ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 41.610833ms iteration = 21 status = {"errors":[],"garbage_collect_requested":[]}
29422024-11-28T02:03:10.528ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2024-11-28 01:53:10.479231029 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
29432024-11-28T02:03:10.528ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 48.794198ms iteration = 11 status = {"expiration":"2024-11-28T01:53:10.479231029Z","pruned":[]}
29442024-11-28T02:03:10.529ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 50.187942ms iteration = 11 status = {"errors":[],"found_port_ok":[]}
29452024-11-28T02:03:10.533ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29462024-11-28T02:03:10.537ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
29472024-11-28T02:03:10.544ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29482024-11-28T02:03:10.544ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-313847b3-d670-4167-9b09-9e7d3dfcb2a4.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-313847b3-d670-4167-9b09-9e7d3dfcb2a4.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1457, tv_nsec: 15121427 } })
29492024-11-28T02:03:10.545ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1103 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc
29502024-11-28T02:03:10.547ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
29512024-11-28T02:03:10.547ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
29522024-11-28T02:03:10.548ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
29532024-11-28T02:03:10.548ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29542024-11-28T02:03:10.551ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
29552024-11-28T02:03:10.551ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
29562024-11-28T02:03:10.553ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29572024-11-28T02:03:10.553ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
29582024-11-28T02:03:10.556ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
29592024-11-28T02:03:10.557ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
29602024-11-28T02:03:10.557ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
29612024-11-28T02:03:10.557ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
29622024-11-28T02:03:10.558ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
29632024-11-28T02:03:10.558ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
29642024-11-28T02:03:10.558ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
29652024-11-28T02:03:10.559ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29662024-11-28T02:03:10.560ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
29672024-11-28T02:03:10.560ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
29682024-11-28T02:03:10.562ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
29692024-11-28T02:03:10.562ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
29702024-11-28T02:03:10.562ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
29712024-11-28T02:03:10.563ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
29722024-11-28T02:03:10.563ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
29732024-11-28T02:03:10.563ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
29742024-11-28T02:03:10.564ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
29752024-11-28T02:03:10.564ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29762024-11-28T02:03:10.565ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
29772024-11-28T02:03:10.565ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29782024-11-28T02:03:10.565ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
29792024-11-28T02:03:10.566ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
29802024-11-28T02:03:10.566ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
29812024-11-28T02:03:10.566ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
29822024-11-28T02:03:10.566ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
29832024-11-28T02:03:10.566ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
29842024-11-28T02:03:10.567ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
29852024-11-28T02:03:10.568ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
29862024-11-28T02:03:10.568ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
29872024-11-28T02:03:10.568ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
29882024-11-28T02:03:10.569ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29892024-11-28T02:03:10.569ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 90.97708ms iteration = 21 status = {"errors":[],"records_set_to_done":[]}
29902024-11-28T02:03:10.569ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
29912024-11-28T02:03:10.569ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
29922024-11-28T02:03:10.569ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
29932024-11-28T02:03:10.570ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29942024-11-28T02:03:10.570ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
29952024-11-28T02:03:10.570ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
29962024-11-28T02:03:10.572ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
29972024-11-28T02:03:10.572ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
29982024-11-28T02:03:10.573ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
29992024-11-28T02:03:10.573ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
30002024-11-28T02:03:10.573ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
30012024-11-28T02:03:10.573ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
30022024-11-28T02:03:10.578ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30032024-11-28T02:03:10.581ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
30042024-11-28T02:03:10.581ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
30052024-11-28T02:03:10.582ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 67ac247f-c934-4c78-857a-fba4fbeb9937, lookup_type: ById(67ac247f-c934-4c78-857a-fba4fbeb9937) } result = Ok(())
30062024-11-28T02:03:10.582ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
30072024-11-28T02:03:10.582ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
30082024-11-28T02:03:10.585ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
30092024-11-28T02:03:10.585ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:171 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
30102024-11-28T02:03:10.585ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 106.615382ms iteration = 11 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
30112024-11-28T02:03:10.586ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
30122024-11-28T02:03:10.586ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
30132024-11-28T02:03:10.586ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
30142024-11-28T02:03:10.587ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
30152024-11-28T02:03:10.587ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 108.408093ms iteration = 21 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
30162024-11-28T02:03:10.587ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
30172024-11-28T02:03:10.587ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
30182024-11-28T02:03:10.588ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30192024-11-28T02:03:10.590ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30202024-11-28T02:03:10.590ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
30212024-11-28T02:03:10.590ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 112.492634ms iteration = 21 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
30222024-11-28T02:03:10.591ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30232024-11-28T02:03:10.591ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
30242024-11-28T02:03:10.591ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
30252024-11-28T02:03:10.591ZDEBGnexus (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(3548509e-2f31-440b-a7d7-e94d858fac66, 0), (ef449652-d42a-4da7-a178-73fc1831a942, 0), (25610650-5f0c-48c8-a64d-10a317bb5bd4, 0)]
30262024-11-28T02:03:10.591ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
30272024-11-28T02:03:10.592ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-313847b3-d670-4167-9b09-9e7d3dfcb2a4.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-313847b3-d670-4167-9b09-9e7d3dfcb2a4.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1457, tv_nsec: 62424418 } })
30282024-11-28T02:03:10.594ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 116.024465ms iteration = 61 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
30292024-11-28T02:03:10.594ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
30302024-11-28T02:03:10.594ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
30312024-11-28T02:03:10.595ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
30322024-11-28T02:03:10.595ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
30332024-11-28T02:03:10.595ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
30342024-11-28T02:03:10.595ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
30352024-11-28T02:03:10.596ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
30362024-11-28T02:03:10.597ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
30372024-11-28T02:03:10.597ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
30382024-11-28T02:03:10.599ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
30392024-11-28T02:03:10.599ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
30402024-11-28T02:03:10.599ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
30412024-11-28T02:03:10.599ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
30422024-11-28T02:03:10.599ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
30432024-11-28T02:03:10.600ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
30442024-11-28T02:03:10.600ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
30452024-11-28T02:03:10.600ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
30462024-11-28T02:03:10.600ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
30472024-11-28T02:03:10.600ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
30482024-11-28T02:03:10.600ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
30492024-11-28T02:03:10.601ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
30502024-11-28T02:03:10.601ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30512024-11-28T02:03:10.603ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 67ac247f-c934-4c78-857a-fba4fbeb9937 original_time_created = 2024-11-28 01:53:03.159475 UTC target_id = 67ac247f-c934-4c78-857a-fba4fbeb9937 time_created = 2024-11-28 01:53:03.159475 UTC
30522024-11-28T02:03:10.603ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 124.539223ms iteration = 61 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"67ac247f-c934-4c78-857a-fba4fbeb9937","time_created":"2024-11-28T01:53:03.159475Z"}
30532024-11-28T02:03:10.606ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
30542024-11-28T02:03:10.606ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
30552024-11-28T02:03:10.606ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
30562024-11-28T02:03:10.610ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30572024-11-28T02:03:10.610ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30582024-11-28T02:03:10.614ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 313847b3-d670-4167-9b09-9e7d3dfcb2a4, lookup_type: ById(313847b3-d670-4167-9b09-9e7d3dfcb2a4) } result = Ok(())
30592024-11-28T02:03:10.614ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
30602024-11-28T02:03:10.614ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
30612024-11-28T02:03:10.615ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
30622024-11-28T02:03:10.615ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
30632024-11-28T02:03:10.615ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
30642024-11-28T02:03:10.623ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 145.07842ms iteration = 21 status = 0
30652024-11-28T02:03:10.629ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
30662024-11-28T02:03:10.629ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
30672024-11-28T02:03:10.629ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 150.735968ms iteration = 21 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
30682024-11-28T02:03:10.629ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 150.86574ms iteration = 21 status = {"errors":[],"requests_created_ok":[],"start_invoked_ok":[]}
30692024-11-28T02:03:10.631ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
30702024-11-28T02:03:10.632ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
30712024-11-28T02:03:10.634ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 155.068063ms iteration = 21 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[]}
30722024-11-28T02:03:10.636ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2024-11-28T01:53:08.890632Z, time_modified: 2024-11-28T01:53:08.890632Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 7e159834-997a-4c03-b9db-1e91d783221c, db_silo: Silo { identity: SiloIdentity { id: 7e159834-997a-4c03-b9db-1e91d783221c, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-11-28T01:53:43.308696Z, time_modified: 2024-11-28T01:53:43.308696Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "bc39d1b3b4bb84a49d6458ed7aedc5bca9e77742c7b35882b64d188c0602d306" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 7e159834-997a-4c03-b9db-1e91d783221c, db_silo: Silo { identity: SiloIdentity { id: 7e159834-997a-4c03-b9db-1e91d783221c, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-11-28T01:53:43.308696Z, time_modified: 2024-11-28T01:53:43.308696Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "bc39d1b3b4bb84a49d6458ed7aedc5bca9e77742c7b35882b64d188c0602d306" }] }) }
30732024-11-28T02:03:10.636ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 158.242679ms iteration = 12 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"7e159834-997a-4c03-b9db-1e91d783221c","tls_certs":["bc39d1b3b4bb84a49d6458ed7aedc5bca9e77742c7b35882b64d188c0602d306"]}},"default_endpoint":{"silo_id":"7e159834-997a-4c03-b9db-1e91d783221c","tls_certs":["bc39d1b3b4bb84a49d6458ed7aedc5bca9e77742c7b35882b64d188c0602d306"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
30742024-11-28T02:03:10.636ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
30752024-11-28T02:03:10.637ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
30762024-11-28T02:03:10.638ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
30772024-11-28T02:03:10.639ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
30782024-11-28T02:03:10.641ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30792024-11-28T02:03:10.642ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
30802024-11-28T02:03:10.642ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
30812024-11-28T02:03:10.642ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:140 found = 0 not_ready_to_be_deleted = 0
30822024-11-28T02:03:10.642ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 163.5097ms iteration = 11 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
30832024-11-28T02:03:10.642ZWARNnexus (BackgroundTasks): instance reincarnation completed with errors background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:103 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0 query_errors = 0 restart_errors = 0
30842024-11-28T02:03:10.642ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 164.004939ms iteration = 11 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
30852024-11-28T02:03:10.645ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
30862024-11-28T02:03:10.645ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
30872024-11-28T02:03:10.645ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
30882024-11-28T02:03:10.647ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
30892024-11-28T02:03:10.647ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
30902024-11-28T02:03:10.656ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
30912024-11-28T02:03:10.656ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 808a324e-c913-4bfc-91a5-d43c961cd511 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
30922024-11-28T02:03:10.656ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
30932024-11-28T02:03:10.656ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
30942024-11-28T02:03:10.657ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
30952024-11-28T02:03:10.657ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
30962024-11-28T02:03:10.657ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "b08fa45c-d170-4c68-b183-9fa8fc0ee9f9", "content-length": "26", "date": "Thu, 28 Nov 2024 02:03:10 GMT"} })
30972024-11-28T02:03:10.657ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
30982024-11-28T02:03:10.659ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30992024-11-28T02:03:10.659ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
31002024-11-28T02:03:10.664ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
31012024-11-28T02:03:10.664ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
31022024-11-28T02:03:10.664ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
31032024-11-28T02:03:10.665ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
31042024-11-28T02:03:10.666ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = a7e6af61-408c-483d-b043-c27ccf55708e dns_zone_name = oxide.test found_so_far = 0 version = 2
31052024-11-28T02:03:10.666ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
31062024-11-28T02:03:10.666ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
31072024-11-28T02:03:10.667ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 808a324e-c913-4bfc-91a5-d43c961cd511 dns_zone_name = control-plane.oxide.internal found_so_far = 51 version = 1
31082024-11-28T02:03:10.667ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
31092024-11-28T02:03:10.667ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2024-11-28 01:53:42.502562 UTC dns_group = internal generation = 1
31102024-11-28T02:03:10.667ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 188.746718ms iteration = 12 status = {"generation":1}
31112024-11-28T02:03:10.668ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31122024-11-28T02:03:10.672ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
31132024-11-28T02:03:10.672ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
31142024-11-28T02:03:10.673ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
31152024-11-28T02:03:10.674ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
31162024-11-28T02:03:10.674ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
31172024-11-28T02:03:10.674ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "f642584c-4408-4aea-baed-58bad529917c", "content-length": "26", "date": "Thu, 28 Nov 2024 02:03:10 GMT"} })
31182024-11-28T02:03:10.674ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
31192024-11-28T02:03:10.675ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = a7e6af61-408c-483d-b043-c27ccf55708e dns_zone_name = oxide.test found_so_far = 1 version = 2
31202024-11-28T02:03:10.675ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
31212024-11-28T02:03:10.675ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2024-11-28 01:53:43.351579 UTC dns_group = external generation = 2
31222024-11-28T02:03:10.675ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 196.629795ms iteration = 12 status = {"generation":2}
31232024-11-28T02:03:10.677ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
31242024-11-28T02:03:10.680ZINFOnexus (SagaRecoverer): listed in-progress sagas count = 0 file = nexus/src/app/background/tasks/saga_recovery.rs:438
31252024-11-28T02:03:10.680ZINFOnexus (BackgroundTasks): observed saga start background_task = saga_recovery file = nexus/saga-recovery/src/recovery.rs:76 saga_id = 4d867d38-0d08-4dad-9311-1a648d600bf0
31262024-11-28T02:03:10.680ZINFOnexus (BackgroundTasks): observed saga start background_task = saga_recovery file = nexus/saga-recovery/src/recovery.rs:76 saga_id = 88ff5102-2f98-4f2b-9a0d-83e5fe910a7f
31272024-11-28T02:03:10.680ZINFOnexus (BackgroundTasks): observed saga start background_task = saga_recovery file = nexus/saga-recovery/src/recovery.rs:76 saga_id = 6e4fd50c-9787-44e1-8f25-8f089a8e1153
31282024-11-28T02:03:10.681ZDEBGnexus (BackgroundTasks): found saga that may be done (will be sure on the next pass) background_task = saga_recovery saga_id = 4d867d38-0d08-4dad-9311-1a648d600bf0
31292024-11-28T02:03:10.681ZDEBGnexus (BackgroundTasks): found saga that may be done (will be sure on the next pass) background_task = saga_recovery saga_id = 6e4fd50c-9787-44e1-8f25-8f089a8e1153
31302024-11-28T02:03:10.684ZDEBGnexus (BackgroundTasks): found saga that may be done (will be sure on the next pass) background_task = saga_recovery saga_id = 88ff5102-2f98-4f2b-9a0d-83e5fe910a7f
31312024-11-28T02:03:10.684ZDEBGnexus (BackgroundTasks): activation complete background_task = saga_recovery elapsed = 205.140333ms iteration = 2 status = {"last_pass":{"Success":{"nfailed":0,"nfound":0,"nrecovered":0,"nremoved":0,"nskipped":0}},"ntotal_failures":0,"ntotal_finished":0,"ntotal_recovered":0,"ntotal_sec_errors_bad_state":0,"ntotal_sec_errors_missing":0,"ntotal_started":3,"recent_failures":[],"recent_recoveries":[]}
31322024-11-28T02:03:10.698ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
31332024-11-28T02:03:10.698ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
31342024-11-28T02:03:10.705ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31352024-11-28T02:03:10.710ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
31362024-11-28T02:03:10.710ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
31372024-11-28T02:03:10.710ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
31382024-11-28T02:03:10.714ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
31392024-11-28T02:03:10.714ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
31402024-11-28T02:03:10.750ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 271.246672ms iteration = 24 status = {"physical_disks_added":0}
31412024-11-28T02:03:10.855ZINFOnexus (BackgroundTasks): VPC route manager sled 313847b3-d670-4167-9b09-9e7d3dfcb2a4 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
31422024-11-28T02:03:10.855ZDEBGnexus (BackgroundTasks): client request SledAgent = 313847b3-d670-4167-9b09-9e7d3dfcb2a4 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
31432024-11-28T02:03:10.855ZDEBGnexus (BackgroundTasks): client response SledAgent = 313847b3-d670-4167-9b09-9e7d3dfcb2a4 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "d5426e49-2057-4f5c-9f29-085420973f32", "content-length": "372", "date": "Thu, 28 Nov 2024 02:03:10 GMT"} })
31442024-11-28T02:03:10.855ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
31452024-11-28T02:03:10.856ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
31462024-11-28T02:03:10.856ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
31472024-11-28T02:03:10.856ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "a14c8139-5408-4fae-93d7-aaeacfe11bfc", "content-length": "1", "date": "Thu, 28 Nov 2024 02:03:10 GMT"} })
31482024-11-28T02:03:10.857ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
31492024-11-28T02:03:10.858ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
31502024-11-28T02:03:10.858ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
31512024-11-28T02:03:10.858ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
31522024-11-28T02:03:10.859ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
31532024-11-28T02:03:10.859ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "fe99f984-f63b-463d-b8d6-a01b1194bdcb", "date": "Thu, 28 Nov 2024 02:03:10 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
31542024-11-28T02:03:10.859ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
31552024-11-28T02:03:10.860ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
31562024-11-28T02:03:10.860ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
31572024-11-28T02:03:10.860ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 381.782051ms iteration = 21 status = null
31582024-11-28T02:03:10.861ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "94cf45cb-f34a-49ed-b132-866182ccafe5", "date": "Thu, 28 Nov 2024 02:03:10 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
31592024-11-28T02:03:10.861ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
31602024-11-28T02:03:10.862ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
31612024-11-28T02:03:10.862ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
31622024-11-28T02:03:10.863ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
31632024-11-28T02:03:10.864ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 385.644638ms iteration = 13 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
31642024-11-28T02:03:10.864ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
31652024-11-28T02:03:10.865ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
31662024-11-28T02:03:10.866ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
31672024-11-28T02:03:10.866ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
31682024-11-28T02:03:10.867ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1111 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc zone_address = fd00:1122:3344:101::2
31692024-11-28T02:03:10.867ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
31702024-11-28T02:03:10.867ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
31712024-11-28T02:03:10.867ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "353d58e8-10a3-462c-825f-e156bb01f78c", "content-length": "26", "date": "Thu, 28 Nov 2024 02:03:10 GMT"} })
31722024-11-28T02:03:10.868ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1116 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
31732024-11-28T02:03:10.868ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
31742024-11-28T02:03:10.869ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc
31752024-11-28T02:03:10.869ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31762024-11-28T02:03:10.872ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
31772024-11-28T02:03:10.872ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
31782024-11-28T02:03:10.872ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
31792024-11-28T02:03:10.874ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
31802024-11-28T02:03:10.874ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
31812024-11-28T02:03:10.876ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
31822024-11-28T02:03:10.876ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
31832024-11-28T02:03:10.877ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
31842024-11-28T02:03:10.878ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
31852024-11-28T02:03:10.879ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
31862024-11-28T02:03:10.881ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
31872024-11-28T02:03:10.881ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
31882024-11-28T02:03:10.882ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
31892024-11-28T02:03:10.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
31902024-11-28T02:03:10.885ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 407.02517ms iteration = 21 status = {}
31912024-11-28T02:03:10.901ZDEBGnexus (BackgroundTasks): begin collection background_task = inventory_collection
31922024-11-28T02:03:10.901ZDEBGnexus (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[fd00:1122:3344:101::2]:12225
31932024-11-28T02:03:10.901ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/ignition
31942024-11-28T02:03:10.901ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "ba246664-fde1-4284-b8fc-99d558f1fee2", "content-length": "476", "date": "Thu, 28 Nov 2024 02:03:10 GMT"} })
31952024-11-28T02:03:10.901ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0
31962024-11-28T02:03:10.901ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "6dd9f24f-ea63-4c24-9d87-4a64f2ff737a", "content-length": "734", "date": "Thu, 28 Nov 2024 02:03:10 GMT"} })
31972024-11-28T02:03:10.901ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=0
31982024-11-28T02:03:10.902ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "125819f3-5b8b-44e3-bed7-b06aa2a45162", "content-length": "111", "date": "Thu, 28 Nov 2024 02:03:10 GMT"} })
31992024-11-28T02:03:10.902ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=1
32002024-11-28T02:03:10.902ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "9fbe8335-f5f1-4ee1-a119-1e6929bf6f8e", "content-length": "111", "date": "Thu, 28 Nov 2024 02:03:10 GMT"} })
32012024-11-28T02:03:10.902ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=0
32022024-11-28T02:03:10.902ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "8bb124a3-a1c2-4f6b-984d-e9dba3031d41", "content-length": "167", "date": "Thu, 28 Nov 2024 02:03:10 GMT"} })
32032024-11-28T02:03:10.902ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=1
32042024-11-28T02:03:10.902ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "2b1f53be-55bd-4aa8-8ad5-69a9d25b8e70", "content-length": "167", "date": "Thu, 28 Nov 2024 02:03:10 GMT"} })
32052024-11-28T02:03:10.902ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=0
32062024-11-28T02:03:10.903ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "11c363b6-1468-4024-ae51-0ccba6ae7ac9", "content-length": "175", "date": "Thu, 28 Nov 2024 02:03:10 GMT"} })
32072024-11-28T02:03:10.903ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=1
32082024-11-28T02:03:10.903ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "4efa23d7-fe61-4c42-8f59-22340a4a66f0", "content-length": "175", "date": "Thu, 28 Nov 2024 02:03:10 GMT"} })
32092024-11-28T02:03:10.903ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cmpa
32102024-11-28T02:03:10.903ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "50b7fb6d-722b-4f31-96af-8249b411d72b", "content-length": "702", "date": "Thu, 28 Nov 2024 02:03:10 GMT"} })
32112024-11-28T02:03:10.903ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
32122024-11-28T02:03:10.903ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "00a46f1e-6fca-4581-8ed7-94bf549c1219", "content-length": "727", "date": "Thu, 28 Nov 2024 02:03:10 GMT"} })
32132024-11-28T02:03:10.903ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
32142024-11-28T02:03:10.904ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "3ffc7367-25a1-4a0b-833d-dfc5627eb1a1", "content-length": "729", "date": "Thu, 28 Nov 2024 02:03:10 GMT"} })
32152024-11-28T02:03:10.904ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
32162024-11-28T02:03:10.904ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "0bc79119-30d0-4089-9509-d1e3a5a39f05", "content-length": "728", "date": "Thu, 28 Nov 2024 02:03:10 GMT"} })
32172024-11-28T02:03:10.904ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0
32182024-11-28T02:03:10.904ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "c3d6841c-39b7-4196-9894-40c56da2523b", "content-length": "722", "date": "Thu, 28 Nov 2024 02:03:10 GMT"} })
32192024-11-28T02:03:10.904ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=0
32202024-11-28T02:03:10.904ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "5da5e0b2-5bae-4c34-b38f-fd59f7519e6c", "content-length": "109", "date": "Thu, 28 Nov 2024 02:03:10 GMT"} })
32212024-11-28T02:03:10.904ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=1
32222024-11-28T02:03:10.905ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "f854fbd4-4c5c-4e04-9dfd-73d7719c2e10", "content-length": "109", "date": "Thu, 28 Nov 2024 02:03:10 GMT"} })
32232024-11-28T02:03:10.905ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=0
32242024-11-28T02:03:10.905ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "22ee33d7-46ed-4eb8-891b-c971ba50903a", "content-length": "169", "date": "Thu, 28 Nov 2024 02:03:10 GMT"} })
32252024-11-28T02:03:10.905ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=1
32262024-11-28T02:03:10.905ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "1e2aa309-89cc-4ceb-928e-349fb664d711", "content-length": "169", "date": "Thu, 28 Nov 2024 02:03:10 GMT"} })
32272024-11-28T02:03:10.905ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=0
32282024-11-28T02:03:10.905ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "b67e641b-2b89-416b-b786-96d80887ef24", "content-length": "178", "date": "Thu, 28 Nov 2024 02:03:10 GMT"} })
32292024-11-28T02:03:10.905ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=1
32302024-11-28T02:03:10.906ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "e1f60431-8bdf-4693-93b0-6e32d6cf5978", "content-length": "178", "date": "Thu, 28 Nov 2024 02:03:10 GMT"} })
32312024-11-28T02:03:10.906ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cmpa
32322024-11-28T02:03:10.906ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "32535e2e-835c-40d6-a1c5-9d0dc8be0e81", "content-length": "702", "date": "Thu, 28 Nov 2024 02:03:10 GMT"} })
32332024-11-28T02:03:10.906ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
32342024-11-28T02:03:10.906ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "ab53f60b-6128-41ed-a926-fb5c68c59bf5", "content-length": "727", "date": "Thu, 28 Nov 2024 02:03:10 GMT"} })
32352024-11-28T02:03:10.906ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
32362024-11-28T02:03:10.906ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "d73d889a-8299-40b1-a589-5fab33cd487a", "content-length": "729", "date": "Thu, 28 Nov 2024 02:03:10 GMT"} })
32372024-11-28T02:03:10.906ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
32382024-11-28T02:03:10.906ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "f54b76b9-39ae-47c2-9b5d-cf22783bced4", "content-length": "728", "date": "Thu, 28 Nov 2024 02:03:10 GMT"} })
32392024-11-28T02:03:10.906ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
32402024-11-28T02:03:10.907ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
32412024-11-28T02:03:10.908ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
32422024-11-28T02:03:10.909ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
32432024-11-28T02:03:10.909ZDEBGnexus (BackgroundTasks): client request SledAgent = 313847b3-d670-4167-9b09-9e7d3dfcb2a4 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
32442024-11-28T02:03:10.909ZDEBGnexus (BackgroundTasks): client response SledAgent = 313847b3-d670-4167-9b09-9e7d3dfcb2a4 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "1bf565dd-5c15-4a3f-9456-f255fcb4f19d", "date": "Thu, 28 Nov 2024 02:03:10 GMT"} })
32452024-11-28T02:03:10.909ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
32462024-11-28T02:03:10.910ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
32472024-11-28T02:03:10.911ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
32482024-11-28T02:03:10.911ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
32492024-11-28T02:03:10.911ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
32502024-11-28T02:03:10.912ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
32512024-11-28T02:03:10.912ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
32522024-11-28T02:03:10.913ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
32532024-11-28T02:03:10.915ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
32542024-11-28T02:03:10.915ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
32552024-11-28T02:03:10.915ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
32562024-11-28T02:03:10.915ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
32572024-11-28T02:03:10.916ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
32582024-11-28T02:03:10.919ZINFOnexus (BackgroundTasks): VPC route manager sled 313847b3-d670-4167-9b09-9e7d3dfcb2a4 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
32592024-11-28T02:03:10.919ZDEBGnexus (BackgroundTasks): client request SledAgent = 313847b3-d670-4167-9b09-9e7d3dfcb2a4 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
32602024-11-28T02:03:10.919ZDEBGnexus (BackgroundTasks): client response SledAgent = 313847b3-d670-4167-9b09-9e7d3dfcb2a4 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "c324a67e-fbed-4957-9124-d41c48df2e1e", "date": "Thu, 28 Nov 2024 02:03:10 GMT"} })
32612024-11-28T02:03:10.921ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 441.924346ms iteration = 22 status = {}
32622024-11-28T02:03:10.922ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
32632024-11-28T02:03:10.922ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
32642024-11-28T02:03:10.924ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
32652024-11-28T02:03:10.927ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
32662024-11-28T02:03:10.927ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
32672024-11-28T02:03:10.927ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
32682024-11-28T02:03:10.929ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
32692024-11-28T02:03:10.929ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
32702024-11-28T02:03:10.931ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
32712024-11-28T02:03:10.931ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
32722024-11-28T02:03:10.932ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
32732024-11-28T02:03:10.934ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
32742024-11-28T02:03:10.934ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "306d4884-8668-4ef8-a0a2-4947055fe8de", "date": "Thu, 28 Nov 2024 02:03:10 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
32752024-11-28T02:03:10.934ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
32762024-11-28T02:03:10.935ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
32772024-11-28T02:03:10.935ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
32782024-11-28T02:03:10.935ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
32792024-11-28T02:03:10.937ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "84bbe91d-c564-4e26-b31a-0819f8b6fdff", "date": "Thu, 28 Nov 2024 02:03:10 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
32802024-11-28T02:03:10.937ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
32812024-11-28T02:03:10.939ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
32822024-11-28T02:03:10.939ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "fc102f9a-889d-4c27-8ac2-222ea2e87465", "date": "Thu, 28 Nov 2024 02:03:10 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
32832024-11-28T02:03:10.939ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
32842024-11-28T02:03:10.939ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
32852024-11-28T02:03:10.941ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
32862024-11-28T02:03:10.941ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
32872024-11-28T02:03:10.942ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
32882024-11-28T02:03:10.942ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 463.154775ms iteration = 13 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
32892024-11-28T02:03:10.943ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
32902024-11-28T02:03:10.943ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
32912024-11-28T02:03:10.945ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
32922024-11-28T02:03:10.945ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
32932024-11-28T02:03:10.948ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
32942024-11-28T02:03:10.948ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
32952024-11-28T02:03:10.950ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
32962024-11-28T02:03:10.950ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
32972024-11-28T02:03:10.951ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
32982024-11-28T02:03:10.951ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
32992024-11-28T02:03:10.953ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
33002024-11-28T02:03:10.956ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
33012024-11-28T02:03:10.956ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
33022024-11-28T02:03:10.956ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
33032024-11-28T02:03:10.958ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
33042024-11-28T02:03:10.958ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
33052024-11-28T02:03:10.959ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
33062024-11-28T02:03:10.959ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
33072024-11-28T02:03:10.961ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
33082024-11-28T02:03:10.964ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
33092024-11-28T02:03:10.964ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
33102024-11-28T02:03:10.964ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
33112024-11-28T02:03:10.966ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
33122024-11-28T02:03:10.967ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
33132024-11-28T02:03:10.968ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
33142024-11-28T02:03:10.970ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
33152024-11-28T02:03:10.970ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
33162024-11-28T02:03:10.970ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
33172024-11-28T02:03:10.972ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
33182024-11-28T02:03:10.972ZDEBGnexus (service-firewall-rule-progator): resolved names for firewall rules background_task = service_firewall_rule_propagation instance_interfaces = {} subnet_interfaces = {Name("nexus"): [NetworkInterface { id: d932a551-47e5-4093-9823-e79bc6fd3354, kind: Service { id: 6392fcec-0540-4c8e-8fb9-a456637e79d1 }, name: Name("nexus-6392fcec-0540-4c8e-8fb9-a456637e79d1"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 167, 213])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: f17d445e-bb27-44c7-a47e-78951fc5205d, kind: Service { id: 2d033909-79ed-49ae-b11d-aeb32740cc17 }, name: Name("nexus-2d033909-79ed-49ae-b11d-aeb32740cc17"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 221, 212])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: fa7a6292-a407-4f82-8188-c5971cd240ed, kind: Service { id: 2469a748-bcd2-4e0d-99cc-f79def0452cc }, name: Name("nexus-2469a748-bcd2-4e0d-99cc-f79def0452cc"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 228, 85])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], Name("external-dns"): [NetworkInterface { id: bdebcae5-93f8-4dda-99a8-d4a7f51c845c, kind: Service { id: 36d4f615-7461-4364-960c-89ece3aaabaf }, name: Name("external-dns-36d4f615-7461-4364-960c-89ece3aaabaf"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 140, 1])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: cc93e5ec-9515-444a-bf62-fb07fd9a5d9d, kind: Service { id: eb274350-c0b0-4bce-9c8a-502880f2c905 }, name: Name("external-dns-eb274350-c0b0-4bce-9c8a-502880f2c905"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 133, 17])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }]} subnet_networks = {Name("nexus"): [V4(Ipv4Network { addr: 172.30.2.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:2::, prefix: 64 })], Name("external-dns"): [V4(Ipv4Network { addr: 172.30.1.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:1::, prefix: 64 })]} vpc_interfaces = {}
33192024-11-28T02:03:10.972ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
33202024-11-28T02:03:10.972ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
33212024-11-28T02:03:10.973ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
33222024-11-28T02:03:10.974ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
33232024-11-28T02:03:10.974ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
33242024-11-28T02:03:10.974ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
33252024-11-28T02:03:10.976ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Fleet result = Ok(())
33262024-11-28T02:03:10.976ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
33272024-11-28T02:03:10.977ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
33282024-11-28T02:03:11.012ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
33292024-11-28T02:03:11.013ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
33302024-11-28T02:03:11.015ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 4bb2986f-2e92-4ac3-9dc1-591b6bb60b3e, rack_id: 92a8ff6e-21d3-4c28-bcad-e323847985dc, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(ea300318-9d62-42ed-80ea-21c86590a166) } rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc switch_location = Switch0
33312024-11-28T02:03:11.015ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
33322024-11-28T02:03:11.015ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
33332024-11-28T02:03:11.029ZDEBGnexus (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[fd00:1122:3344:101::1]:12345
33342024-11-28T02:03:11.029ZDEBGnexus (BackgroundTasks): client request SledAgent = http://[fd00:1122:3344:101::1]:12345 background_task = inventory_collection body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/inventory
33352024-11-28T02:03:11.050ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
33362024-11-28T02:03:11.050ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "a1be6e04-4f8c-4f46-be36-3e7983be4b71", "content-length": "134", "date": "Thu, 28 Nov 2024 02:03:10 GMT"} })
33372024-11-28T02:03:11.050ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1746 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
33382024-11-28T02:03:11.050ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1773 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = ea300318-9d62-42ed-80ea-21c86590a166
33392024-11-28T02:03:11.050ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc routes = {Switch0: {}}
33402024-11-28T02:03:11.050ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
33412024-11-28T02:03:11.050ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
33422024-11-28T02:03:11.050ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc routes = {}
33432024-11-28T02:03:11.050ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
33442024-11-28T02:03:11.051ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
33452024-11-28T02:03:11.051ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc
33462024-11-28T02:03:11.051ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
33472024-11-28T02:03:11.052ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
33482024-11-28T02:03:11.053ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
33492024-11-28T02:03:11.054ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "33a51113-9851-4455-868c-d15a90296ba9", "content-length": "2", "date": "Thu, 28 Nov 2024 02:03:10 GMT"} })
33502024-11-28T02:03:11.054ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
33512024-11-28T02:03:11.054ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "575e15ca-a5ce-4a84-b90f-5964ff22b625", "content-length": "52", "date": "Thu, 28 Nov 2024 02:03:11 GMT"} })
33522024-11-28T02:03:11.054ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc switch_location = Switch0
33532024-11-28T02:03:11.054ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
33542024-11-28T02:03:11.074ZDEBGnexus (service-firewall-rule-progator): Allowlist for user-facing services is set to allow any inbound traffic. Existing VPC firewall rules will not be modified. background_task = service_firewall_rule_propagation
33552024-11-28T02:03:11.074ZDEBGnexus (service-firewall-rule-progator): resolved firewall rules for sled agents background_task = service_firewall_rule_propagation sled_agent_rules = [ResolvedVpcFirewallRule { action: Allow, direction: Inbound, filter_hosts: None, filter_ports: Some([L4PortRange("53")]), filter_protocols: Some([Udp]), priority: 65534, status: Enabled, targets: [NetworkInterface { id: bdebcae5-93f8-4dda-99a8-d4a7f51c845c, kind: Service { id: 36d4f615-7461-4364-960c-89ece3aaabaf }, name: Name("external-dns-36d4f615-7461-4364-960c-89ece3aaabaf"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 140, 1])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: cc93e5ec-9515-444a-bf62-fb07fd9a5d9d, kind: Service { id: eb274350-c0b0-4bce-9c8a-502880f2c905 }, name: Name("external-dns-eb274350-c0b0-4bce-9c8a-502880f2c905"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 133, 17])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }] }, ResolvedVpcFirewallRule { action: Allow, direction: Inbound, filter_hosts: None, filter_ports: Some([L4PortRange("80"), L4PortRange("443")]), filter_protocols: Some([Tcp]), priority: 65534, status: Enabled, targets: [NetworkInterface { id: d932a551-47e5-4093-9823-e79bc6fd3354, kind: Service { id: 6392fcec-0540-4c8e-8fb9-a456637e79d1 }, name: Name("nexus-6392fcec-0540-4c8e-8fb9-a456637e79d1"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 167, 213])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: f17d445e-bb27-44c7-a47e-78951fc5205d, kind: Service { id: 2d033909-79ed-49ae-b11d-aeb32740cc17 }, name: Name("nexus-2d033909-79ed-49ae-b11d-aeb32740cc17"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 221, 212])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: fa7a6292-a407-4f82-8188-c5971cd240ed, kind: Service { id: 2469a748-bcd2-4e0d-99cc-f79def0452cc }, name: Name("nexus-2469a748-bcd2-4e0d-99cc-f79def0452cc"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 228, 85])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }] }]
33562024-11-28T02:03:11.074ZDEBGnexus (service-firewall-rule-progator): resolved 2 rules for sleds background_task = service_firewall_rule_propagation
33572024-11-28T02:03:11.085ZDEBGnexus (BackgroundTasks): client request SledAgent = 313847b3-d670-4167-9b09-9e7d3dfcb2a4 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
33582024-11-28T02:03:11.093ZDEBGnexus (BackgroundTasks): client response SledAgent = 313847b3-d670-4167-9b09-9e7d3dfcb2a4 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "84894454-e029-416b-862c-b433cc41c204", "content-length": "691", "date": "Thu, 28 Nov 2024 02:03:11 GMT"} })
33592024-11-28T02:03:11.093ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
33602024-11-28T02:03:11.093ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
33612024-11-28T02:03:11.093ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
33622024-11-28T02:03:11.094ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 615.790676ms iteration = 21 status = {}
33632024-11-28T02:03:11.143ZDEBGnexus (service-firewall-rule-progator): resolved sleds for vpc oxide-services background_task = service_firewall_rule_propagation vpc_to_sled = [Sled { identity: SledIdentity { id: 313847b3-d670-4167-9b09-9e7d3dfcb2a4, time_created: 2024-11-28T01:53:42.307381Z, time_modified: 2024-11-28T01:53:42.307381Z }, time_deleted: None, rcgen: Generation(Generation(19)), rack_id: 92a8ff6e-21d3-4c28-bcad-e323847985dc, is_scrimlet: true, serial_number: "sock", part_number: "i86pc", revision: SqlU32(0), usable_hardware_threads: SqlU32(48), usable_physical_ram: ByteCount(ByteCount(102952026112)), reservoir_size: ByteCount(ByteCount(51474595840)), ip: fd00:1122:3344:101::1, port: SqlU16(12345), last_used_address: fd00:1122:3344:101::ffff, policy: InService, state: Active, sled_agent_gen: Generation(Generation(1)) }]
33642024-11-28T02:03:11.143ZDEBGnexus (service-firewall-rule-progator): sending firewall rules to sled agents background_task = service_firewall_rule_propagation
33652024-11-28T02:03:11.143ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
33662024-11-28T02:03:11.143ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
33672024-11-28T02:03:11.145ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
33682024-11-28T02:03:11.145ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
33692024-11-28T02:03:11.146ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
33702024-11-28T02:03:11.149ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Sled { parent: Fleet, key: 313847b3-d670-4167-9b09-9e7d3dfcb2a4, lookup_type: ById(313847b3-d670-4167-9b09-9e7d3dfcb2a4) } result = Ok(())
33712024-11-28T02:03:11.167ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "63130535-6450-40f3-9270-a9d6ca642890", "date": "Thu, 28 Nov 2024 02:03:11 GMT"} })
33722024-11-28T02:03:11.167ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc switch = Switch0
33732024-11-28T02:03:11.167ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
33742024-11-28T02:03:11.172ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "21eaced4-24e4-4eba-a2b7-544d89855c87", "content-length": "682", "date": "Thu, 28 Nov 2024 02:03:11 GMT"} })
33752024-11-28T02:03:11.172ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
33762024-11-28T02:03:11.172ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
33772024-11-28T02:03:11.180ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
33782024-11-28T02:03:11.181ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
33792024-11-28T02:03:11.185ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
33802024-11-28T02:03:11.186ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
33812024-11-28T02:03:11.187ZDEBGnexus (service-firewall-rule-progator): client request SledAgent = 313847b3-d670-4167-9b09-9e7d3dfcb2a4 background_task = service_firewall_rule_propagation body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules
33822024-11-28T02:03:11.188ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
33832024-11-28T02:03:11.188ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
33842024-11-28T02:03:11.190ZDEBGnexus (service-firewall-rule-progator): client response SledAgent = 313847b3-d670-4167-9b09-9e7d3dfcb2a4 background_task = service_firewall_rule_propagation result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules", status: 204, headers: {"x-request-id": "3f2eb5b0-76f3-4a6d-a711-e4f904b2d20e", "date": "Thu, 28 Nov 2024 02:03:11 GMT"} })
33852024-11-28T02:03:11.192ZINFOnexus (service-firewall-rule-progator): updated firewall rules on 1 sleds background_task = service_firewall_rule_propagation file = nexus/networking/src/firewall_rules.rs:496
33862024-11-28T02:03:11.192ZDEBGnexus (service-firewall-rule-progator): successfully propagated service firewall rules background_task = service_firewall_rule_propagation elapsed = 712.891452ms
33872024-11-28T02:03:11.192ZDEBGnexus (BackgroundTasks): activation complete background_task = service_firewall_rule_propagation elapsed = 712.899633ms iteration = 3 status = {}
33882024-11-28T02:03:11.196ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 717.90954ms iteration = 21 status = {}
33892024-11-28T02:03:11.346ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:40402 req_id = e2e613c0-58ab-4f80-ae70-262c80fc91f4 roles = RoleSet { roles: {} } uri = /nat/ipv4/changeset/6?limit=100
33902024-11-28T02:03:11.346ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:40402 req_id = e2e613c0-58ab-4f80-ae70-262c80fc91f4 resource = Database result = Ok(()) uri = /nat/ipv4/changeset/6?limit=100
33912024-11-28T02:03:11.349ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 3039 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:40402 req_id = e2e613c0-58ab-4f80-ae70-262c80fc91f4 response_code = 200 uri = /nat/ipv4/changeset/6?limit=100
33922024-11-28T02:03:11.366ZDEBGnexus (BackgroundTasks): client response SledAgent = http://[fd00:1122:3344:101::1]:12345 background_task = inventory_collection result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "3657dfb0-62d0-4c74-a442-824f1921d11d", "content-length": "30800", "date": "Thu, 28 Nov 2024 02:03:11 GMT"} })
33932024-11-28T02:03:11.369ZDEBGnexus (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
33942024-11-28T02:03:11.369ZDEBGnexus (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
33952024-11-28T02:03:11.369ZDEBGnexus (BackgroundTasks): finished collection background_task = inventory_collection
33962024-11-28T02:03:11.371ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
33972024-11-28T02:03:11.371ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
33982024-11-28T02:03:11.372ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
33992024-11-28T02:03:11.374ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
34002024-11-28T02:03:11.374ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
34012024-11-28T02:03:11.374ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
34022024-11-28T02:03:11.438ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = bd61d2fb-ef92-44f1-9045-45ae0bcd04fd roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
34032024-11-28T02:03:11.438ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = bd61d2fb-ef92-44f1-9045-45ae0bcd04fd resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
34042024-11-28T02:03:11.440ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2073 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = bd61d2fb-ef92-44f1-9045-45ae0bcd04fd response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
34052024-11-28T02:03:11.556ZINFOnexus (BackgroundTasks): inserted inventory collection background_task = inventory_collection collection_id = 8c324d33-2793-4c58-894a-9cbd2cc1db1b file = nexus/db-queries/src/db/datastore/inventory.rs:1006
34062024-11-28T02:03:11.556ZDEBGnexus (BackgroundTasks): inventory collection complete background_task = inventory_collection collection_id = 8c324d33-2793-4c58-894a-9cbd2cc1db1b time_started = 2024-11-28 02:03:10.901075 UTC
34072024-11-28T02:03:11.556ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 1.07835392s iteration = 4 status = {"collection_id":"8c324d33-2793-4c58-894a-9cbd2cc1db1b","time_done":"2024-11-28 02:03:11.369067 UTC","time_started":"2024-11-28 02:03:10.901075 UTC"}
34082024-11-28T02:03:11.556ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 25 reason = Dependency
34092024-11-28T02:03:11.556ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
34102024-11-28T02:03:11.557ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
34112024-11-28T02:03:11.558ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
34122024-11-28T02:03:11.561ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
34132024-11-28T02:03:11.561ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
34142024-11-28T02:03:11.561ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
34152024-11-28T02:03:11.563ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
34162024-11-28T02:03:11.563ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
34172024-11-28T02:03:11.565ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
34182024-11-28T02:03:11.567ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
34192024-11-28T02:03:11.567ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
34202024-11-28T02:03:11.567ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
34212024-11-28T02:03:11.577ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 21.001695ms iteration = 25 status = {"physical_disks_added":0}
34222024-11-28T02:03:11.840ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::b]:12221 remote_addr = [fd00:1122:3344:101::2]:55448
34232024-11-28T02:03:11.840ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:55448 req_id = 04b31ab6-c089-4baf-be09-4878c2cb5c10 roles = RoleSet { roles: {} } uri = /metrics/producers
34242024-11-28T02:03:11.840ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:55448 req_id = 04b31ab6-c089-4baf-be09-4878c2cb5c10 resource = Database result = Ok(()) uri = /metrics/producers
34252024-11-28T02:03:11.857ZINFOnexus: assigned collector to new producer collector_id = 119eed64-2e24-4d5f-8c42-be6cd8bec8b5 file = nexus/src/app/oximeter.rs:76 producer_id = 4ea85f4f-c0ac-4d9e-ae1a-e4f649a8809c
34262024-11-28T02:03:11.857ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 17393 local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:55448 req_id = 04b31ab6-c089-4baf-be09-4878c2cb5c10 response_code = 201 uri = /metrics/producers
34272024-11-28T02:03:12.044ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::b]:12221 remote_addr = [fd00:1122:3344:101::1]:60562
34282024-11-28T02:03:12.044ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::1]:60562 req_id = d6609972-fb28-4b88-935e-d05879fb44ec roles = RoleSet { roles: {} } uri = /metrics/producers
34292024-11-28T02:03:12.044ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::1]:60562 req_id = d6609972-fb28-4b88-935e-d05879fb44ec resource = Database result = Ok(()) uri = /metrics/producers
34302024-11-28T02:03:12.086ZINFOnexus: assigned collector to new producer collector_id = 119eed64-2e24-4d5f-8c42-be6cd8bec8b5 file = nexus/src/app/oximeter.rs:76 producer_id = 7d8f995c-f158-4cd9-9fae-a9ac17bfe5ed
34312024-11-28T02:03:12.086ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 42440 local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::1]:60562 req_id = d6609972-fb28-4b88-935e-d05879fb44ec response_code = 201 uri = /metrics/producers
34322024-11-28T02:03:12.474ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = d3a70057-4e3e-426a-a3e4-3283db907380 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
34332024-11-28T02:03:12.475ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = d3a70057-4e3e-426a-a3e4-3283db907380 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
34342024-11-28T02:03:12.477ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2532 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = d3a70057-4e3e-426a-a3e4-3283db907380 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
34352024-11-28T02:03:12.634ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::b]:12221 remote_addr = [fd00:1122:3344:101::2]:33283
34362024-11-28T02:03:12.634ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:33283 req_id = 33e63761-024c-467c-b418-aca836900878 roles = RoleSet { roles: {} } uri = /metrics/producers
34372024-11-28T02:03:12.635ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:33283 req_id = 33e63761-024c-467c-b418-aca836900878 resource = Database result = Ok(()) uri = /metrics/producers
34382024-11-28T02:03:12.670ZINFOnexus: assigned collector to new producer collector_id = 119eed64-2e24-4d5f-8c42-be6cd8bec8b5 file = nexus/src/app/oximeter.rs:76 producer_id = 37e16e5f-8832-402e-b7dd-2f5b87405438
34392024-11-28T02:03:12.670ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 36174 local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:33283 req_id = 33e63761-024c-467c-b418-aca836900878 response_code = 201 uri = /metrics/producers
34402024-11-28T02:03:13.496ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 2cac76b4-f13f-4f2f-8ab5-df9dcaa37276 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
34412024-11-28T02:03:13.496ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 2cac76b4-f13f-4f2f-8ab5-df9dcaa37276 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
34422024-11-28T02:03:13.498ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2072 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 2cac76b4-f13f-4f2f-8ab5-df9dcaa37276 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
34432024-11-28T02:03:14.194ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1235 local_addr = [fd00:1122:3344:101::b]:55621 method = GET remote_addr = [fd00:1122:3344:101::d]:56417 req_id = ce54ed0e-89ce-4ae9-ab7a-f70f6fd8d35b response_code = 200 uri = /2469a748-bcd2-4e0d-99cc-f79def0452cc
34442024-11-28T02:03:14.518ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 015f5f8b-826d-4aa1-b424-95ecbdea796f roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
34452024-11-28T02:03:14.518ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 015f5f8b-826d-4aa1-b424-95ecbdea796f resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
34462024-11-28T02:03:14.520ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2400 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 015f5f8b-826d-4aa1-b424-95ecbdea796f response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
34472024-11-28T02:03:15.540ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 5ea7ce97-d147-46d2-8127-3da6dc6c41ce roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
34482024-11-28T02:03:15.540ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 5ea7ce97-d147-46d2-8127-3da6dc6c41ce resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
34492024-11-28T02:03:15.542ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2184 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 5ea7ce97-d147-46d2-8127-3da6dc6c41ce response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
34502024-11-28T02:03:16.561ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = d19c954a-b889-40f8-9423-2500f1b61e46 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
34512024-11-28T02:03:16.561ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = d19c954a-b889-40f8-9423-2500f1b61e46 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
34522024-11-28T02:03:16.563ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2316 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = d19c954a-b889-40f8-9423-2500f1b61e46 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
34532024-11-28T02:03:16.885ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::b]:12221 remote_addr = [fd00:1122:3344:101::2]:64571
34542024-11-28T02:03:16.885ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:64571 req_id = 964ae5af-069d-44ff-98e4-950d10fc261f roles = RoleSet { roles: {} } uri = /metrics/producers
34552024-11-28T02:03:16.886ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:64571 req_id = 964ae5af-069d-44ff-98e4-950d10fc261f resource = Database result = Ok(()) uri = /metrics/producers
34562024-11-28T02:03:16.896ZINFOnexus: assigned collector to new producer collector_id = 119eed64-2e24-4d5f-8c42-be6cd8bec8b5 file = nexus/src/app/oximeter.rs:76 producer_id = 27b44fd9-9dc4-42a5-82e8-5084c0434475
34572024-11-28T02:03:16.896ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 11165 local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:64571 req_id = 964ae5af-069d-44ff-98e4-950d10fc261f response_code = 201 uri = /metrics/producers
34582024-11-28T02:03:17.582ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 1c929d8a-899e-4335-9b30-01273e74472d roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
34592024-11-28T02:03:17.582ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 1c929d8a-899e-4335-9b30-01273e74472d resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
34602024-11-28T02:03:17.584ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2255 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 1c929d8a-899e-4335-9b30-01273e74472d response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
34612024-11-28T02:03:18.604ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 0b940f2d-924b-43a3-9518-469537a6b44e roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
34622024-11-28T02:03:18.604ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 0b940f2d-924b-43a3-9518-469537a6b44e resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
34632024-11-28T02:03:18.606ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1768 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 0b940f2d-924b-43a3-9518-469537a6b44e response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
34642024-11-28T02:03:19.625ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = f95a5f51-7fa3-4e50-b885-62356bcfedea roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
34652024-11-28T02:03:19.625ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = f95a5f51-7fa3-4e50-b885-62356bcfedea resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
34662024-11-28T02:03:19.627ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2393 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = f95a5f51-7fa3-4e50-b885-62356bcfedea response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
34672024-11-28T02:03:20.478ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 62 reason = Timeout
34682024-11-28T02:03:20.478ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
34692024-11-28T02:03:20.478ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 62 reason = Timeout
34702024-11-28T02:03:20.478ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
34712024-11-28T02:03:20.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
34722024-11-28T02:03:20.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
34732024-11-28T02:03:20.481ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
34742024-11-28T02:03:20.483ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
34752024-11-28T02:03:20.483ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
34762024-11-28T02:03:20.483ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
34772024-11-28T02:03:20.485ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
34782024-11-28T02:03:20.486ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
34792024-11-28T02:03:20.487ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
34802024-11-28T02:03:20.491ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 67ac247f-c934-4c78-857a-fba4fbeb9937, lookup_type: ById(67ac247f-c934-4c78-857a-fba4fbeb9937) } result = Ok(())
34812024-11-28T02:03:20.491ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
34822024-11-28T02:03:20.491ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
34832024-11-28T02:03:20.503ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 67ac247f-c934-4c78-857a-fba4fbeb9937 original_time_created = 2024-11-28 01:53:03.159475 UTC target_id = 67ac247f-c934-4c78-857a-fba4fbeb9937 time_created = 2024-11-28 01:53:03.159475 UTC
34842024-11-28T02:03:20.503ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 24.839072ms iteration = 62 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"67ac247f-c934-4c78-857a-fba4fbeb9937","time_created":"2024-11-28T01:53:03.159475Z"}
34852024-11-28T02:03:20.508ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
34862024-11-28T02:03:20.508ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
34872024-11-28T02:03:20.536ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 57.274615ms iteration = 62 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
34882024-11-28T02:03:20.648ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 261134c4-9a31-4b6a-aac9-bcf6e5c33fe0 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
34892024-11-28T02:03:20.648ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 261134c4-9a31-4b6a-aac9-bcf6e5c33fe0 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
34902024-11-28T02:03:20.650ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2251 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 261134c4-9a31-4b6a-aac9-bcf6e5c33fe0 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
34912024-11-28T02:03:21.668ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 8be43d55-c3c4-4ebb-a08c-c24af5fc078d roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
34922024-11-28T02:03:21.668ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 8be43d55-c3c4-4ebb-a08c-c24af5fc078d resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
34932024-11-28T02:03:21.670ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1958 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 8be43d55-c3c4-4ebb-a08c-c24af5fc078d response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
34942024-11-28T02:03:22.691ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 8f07d8e0-b38d-4ebf-b029-4a25d230350f roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
34952024-11-28T02:03:22.691ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 8f07d8e0-b38d-4ebf-b029-4a25d230350f resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
34962024-11-28T02:03:22.693ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2182 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 8f07d8e0-b38d-4ebf-b029-4a25d230350f response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
34972024-11-28T02:03:23.711ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 3a4dfdec-3910-4418-8264-71444a5898b3 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
34982024-11-28T02:03:23.712ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 3a4dfdec-3910-4418-8264-71444a5898b3 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
34992024-11-28T02:03:23.714ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2675 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 3a4dfdec-3910-4418-8264-71444a5898b3 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
35002024-11-28T02:03:24.186ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:55495 req_id = f8292675-2cc3-4a7b-bd7f-1a501a19c997 roles = RoleSet { roles: {} } uri = /metrics/collectors/119eed64-2e24-4d5f-8c42-be6cd8bec8b5/producers?sort_by=id_ascending
35012024-11-28T02:03:24.186ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:55495 req_id = f8292675-2cc3-4a7b-bd7f-1a501a19c997 resource = Database result = Ok(()) uri = /metrics/collectors/119eed64-2e24-4d5f-8c42-be6cd8bec8b5/producers?sort_by=id_ascending
35022024-11-28T02:03:24.188ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2382 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:55495 req_id = f8292675-2cc3-4a7b-bd7f-1a501a19c997 response_code = 200 uri = /metrics/collectors/119eed64-2e24-4d5f-8c42-be6cd8bec8b5/producers?sort_by=id_ascending
35032024-11-28T02:03:24.188ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:55495 req_id = f1696447-8f2a-47fb-abf3-72f2985ed76c roles = RoleSet { roles: {} } uri = /metrics/collectors/119eed64-2e24-4d5f-8c42-be6cd8bec8b5/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJjNDM0YjQ0OC01MTk5LTQ2NmUtYWYzMi00ZjYyOWUzMWM3YjEifX0%3D
35042024-11-28T02:03:24.189ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:55495 req_id = f1696447-8f2a-47fb-abf3-72f2985ed76c resource = Database result = Ok(()) uri = /metrics/collectors/119eed64-2e24-4d5f-8c42-be6cd8bec8b5/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJjNDM0YjQ0OC01MTk5LTQ2NmUtYWYzMi00ZjYyOWUzMWM3YjEifX0%3D
35052024-11-28T02:03:24.191ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2460 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:55495 req_id = f1696447-8f2a-47fb-abf3-72f2985ed76c response_code = 200 uri = /metrics/collectors/119eed64-2e24-4d5f-8c42-be6cd8bec8b5/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJjNDM0YjQ0OC01MTk5LTQ2NmUtYWYzMi00ZjYyOWUzMWM3YjEifX0%3D
35062024-11-28T02:03:24.194ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1248 local_addr = [fd00:1122:3344:101::b]:55621 method = GET remote_addr = [fd00:1122:3344:101::d]:56417 req_id = a8fd2fb1-50d6-436c-b84c-eef61181b069 response_code = 200 uri = /2469a748-bcd2-4e0d-99cc-f79def0452cc
35072024-11-28T02:03:24.738ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = d5c0414a-36ed-4d22-9ff4-ab5699c759e3 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
35082024-11-28T02:03:24.739ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = d5c0414a-36ed-4d22-9ff4-ab5699c759e3 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
35092024-11-28T02:03:24.740ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2028 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = d5c0414a-36ed-4d22-9ff4-ab5699c759e3 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
35102024-11-28T02:03:25.760ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = b64cf51a-f229-4afe-a717-e486d7b39a8a roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
35112024-11-28T02:03:25.760ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = b64cf51a-f229-4afe-a717-e486d7b39a8a resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
35122024-11-28T02:03:25.762ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1997 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = b64cf51a-f229-4afe-a717-e486d7b39a8a response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
35132024-11-28T02:03:26.781ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 48dac151-1d7a-405e-82b0-050964030a0e roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
35142024-11-28T02:03:26.781ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 48dac151-1d7a-405e-82b0-050964030a0e resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
35152024-11-28T02:03:26.783ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2018 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 48dac151-1d7a-405e-82b0-050964030a0e response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
35162024-11-28T02:03:26.802ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::b]:12221 remote_addr = [fd00:1122:3344:101::2]:35128
35172024-11-28T02:03:26.802ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:35128 req_id = 09353599-0896-4879-bd0b-8127e0022eb3 roles = RoleSet { roles: {} } uri = /metrics/producers
35182024-11-28T02:03:26.803ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:35128 req_id = 09353599-0896-4879-bd0b-8127e0022eb3 resource = Database result = Ok(()) uri = /metrics/producers
35192024-11-28T02:03:26.840ZINFOnexus: assigned collector to new producer collector_id = 119eed64-2e24-4d5f-8c42-be6cd8bec8b5 file = nexus/src/app/oximeter.rs:76 producer_id = c434b448-5199-466e-af32-4f629e31c7b1
35202024-11-28T02:03:26.840ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 37833 local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:35128 req_id = 09353599-0896-4879-bd0b-8127e0022eb3 response_code = 201 uri = /metrics/producers
35212024-11-28T02:03:27.802ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = d4829417-c1fa-49e1-a04f-d7907dfac753 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
35222024-11-28T02:03:27.803ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = d4829417-c1fa-49e1-a04f-d7907dfac753 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
35232024-11-28T02:03:27.804ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2006 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = d4829417-c1fa-49e1-a04f-d7907dfac753 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
35242024-11-28T02:03:28.823ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = a12c720a-71d9-421d-ab4a-c7746433f59b roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
35252024-11-28T02:03:28.823ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = a12c720a-71d9-421d-ab4a-c7746433f59b resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
35262024-11-28T02:03:28.825ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2507 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = a12c720a-71d9-421d-ab4a-c7746433f59b response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
35272024-11-28T02:03:29.846ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 7f957e99-240c-4608-8c59-df56b2edc5fc roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
35282024-11-28T02:03:29.847ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 7f957e99-240c-4608-8c59-df56b2edc5fc resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
35292024-11-28T02:03:29.848ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2090 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 7f957e99-240c-4608-8c59-df56b2edc5fc response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
35302024-11-28T02:03:30.479ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 63 reason = Timeout
35312024-11-28T02:03:30.479ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 63 reason = Timeout
35322024-11-28T02:03:30.479ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
35332024-11-28T02:03:30.479ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
35342024-11-28T02:03:30.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
35352024-11-28T02:03:30.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
35362024-11-28T02:03:30.480ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
35372024-11-28T02:03:30.481ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
35382024-11-28T02:03:30.481ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
35392024-11-28T02:03:30.484ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
35402024-11-28T02:03:30.484ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
35412024-11-28T02:03:30.484ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
35422024-11-28T02:03:30.486ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
35432024-11-28T02:03:30.486ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
35442024-11-28T02:03:30.487ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
35452024-11-28T02:03:30.490ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 67ac247f-c934-4c78-857a-fba4fbeb9937, lookup_type: ById(67ac247f-c934-4c78-857a-fba4fbeb9937) } result = Ok(())
35462024-11-28T02:03:30.490ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
35472024-11-28T02:03:30.490ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
35482024-11-28T02:03:30.499ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 67ac247f-c934-4c78-857a-fba4fbeb9937 original_time_created = 2024-11-28 01:53:03.159475 UTC target_id = 67ac247f-c934-4c78-857a-fba4fbeb9937 time_created = 2024-11-28 01:53:03.159475 UTC
35492024-11-28T02:03:30.499ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 20.506326ms iteration = 63 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"67ac247f-c934-4c78-857a-fba4fbeb9937","time_created":"2024-11-28T01:53:03.159475Z"}
35502024-11-28T02:03:30.509ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 30.625872ms iteration = 63 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
35512024-11-28T02:03:30.867ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 7a39e854-b3a8-4f5a-b5bf-fa47410d4620 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
35522024-11-28T02:03:30.868ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 7a39e854-b3a8-4f5a-b5bf-fa47410d4620 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
35532024-11-28T02:03:30.870ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2131 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 7a39e854-b3a8-4f5a-b5bf-fa47410d4620 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
35542024-11-28T02:03:31.889ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 92ba3437-7167-4f92-b174-135c92fe40ac roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
35552024-11-28T02:03:31.890ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 92ba3437-7167-4f92-b174-135c92fe40ac resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
35562024-11-28T02:03:31.891ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2031 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 92ba3437-7167-4f92-b174-135c92fe40ac response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
35572024-11-28T02:03:32.910ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 8592775e-affa-475b-b188-6a92eb2532a9 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
35582024-11-28T02:03:32.911ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 8592775e-affa-475b-b188-6a92eb2532a9 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
35592024-11-28T02:03:32.912ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2109 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 8592775e-affa-475b-b188-6a92eb2532a9 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
35602024-11-28T02:03:33.931ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = fd4f2984-adc2-472f-9ed1-d91ef13ffd5d roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
35612024-11-28T02:03:33.931ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = fd4f2984-adc2-472f-9ed1-d91ef13ffd5d resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
35622024-11-28T02:03:33.933ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1724 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = fd4f2984-adc2-472f-9ed1-d91ef13ffd5d response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
35632024-11-28T02:03:34.195ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1606 local_addr = [fd00:1122:3344:101::b]:55621 method = GET remote_addr = [fd00:1122:3344:101::d]:56417 req_id = 66cb0d8e-c277-451d-80f1-80da39c7e230 response_code = 200 uri = /2469a748-bcd2-4e0d-99cc-f79def0452cc
35642024-11-28T02:03:34.951ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 7159af9f-3752-4c19-aab3-344b83e3f728 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
35652024-11-28T02:03:34.952ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 7159af9f-3752-4c19-aab3-344b83e3f728 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
35662024-11-28T02:03:34.953ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2346 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 7159af9f-3752-4c19-aab3-344b83e3f728 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
35672024-11-28T02:03:35.973ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 648f7c4b-6a94-466a-a3b4-0ec4779e762e roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
35682024-11-28T02:03:35.974ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 648f7c4b-6a94-466a-a3b4-0ec4779e762e resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
35692024-11-28T02:03:35.975ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2216 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 648f7c4b-6a94-466a-a3b4-0ec4779e762e response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
35702024-11-28T02:03:36.993ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = ae7ef8c6-32c7-436a-9a3a-f26c4ad72027 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
35712024-11-28T02:03:36.994ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = ae7ef8c6-32c7-436a-9a3a-f26c4ad72027 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
35722024-11-28T02:03:36.995ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1894 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = ae7ef8c6-32c7-436a-9a3a-f26c4ad72027 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
35732024-11-28T02:03:38.014ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 91b1d382-8688-49fd-9350-bde5a92e10f8 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
35742024-11-28T02:03:38.014ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 91b1d382-8688-49fd-9350-bde5a92e10f8 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
35752024-11-28T02:03:38.016ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2353 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 91b1d382-8688-49fd-9350-bde5a92e10f8 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
35762024-11-28T02:03:39.036ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 51b26066-a4ff-495a-9475-867cfe75577a roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
35772024-11-28T02:03:39.036ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 51b26066-a4ff-495a-9475-867cfe75577a resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
35782024-11-28T02:03:39.038ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2343 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 51b26066-a4ff-495a-9475-867cfe75577a response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
35792024-11-28T02:03:40.058ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 96f21f07-fe60-451f-a21e-6bf8e082efc0 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
35802024-11-28T02:03:40.058ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 96f21f07-fe60-451f-a21e-6bf8e082efc0 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
35812024-11-28T02:03:40.060ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2225 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 96f21f07-fe60-451f-a21e-6bf8e082efc0 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
35822024-11-28T02:03:40.478ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 64 reason = Timeout
35832024-11-28T02:03:40.478ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 22 reason = Timeout
35842024-11-28T02:03:40.478ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
35852024-11-28T02:03:40.478ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
35862024-11-28T02:03:40.478ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 64 reason = Timeout
35872024-11-28T02:03:40.478ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 22 reason = Timeout
35882024-11-28T02:03:40.478ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
35892024-11-28T02:03:40.478ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 22 reason = Timeout
35902024-11-28T02:03:40.478ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
35912024-11-28T02:03:40.478ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
35922024-11-28T02:03:40.478ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
35932024-11-28T02:03:40.478ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 22 reason = Timeout
35942024-11-28T02:03:40.478ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 22 reason = Timeout
35952024-11-28T02:03:40.478ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
35962024-11-28T02:03:40.478ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
35972024-11-28T02:03:40.478ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 22 reason = Timeout
35982024-11-28T02:03:40.478ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
35992024-11-28T02:03:40.478ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 23 reason = Timeout
36002024-11-28T02:03:40.478ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 22 reason = Timeout
36012024-11-28T02:03:40.478ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
36022024-11-28T02:03:40.478ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
36032024-11-28T02:03:40.478ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 22 reason = Timeout
36042024-11-28T02:03:40.478ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 22 reason = Timeout
36052024-11-28T02:03:40.478ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
36062024-11-28T02:03:40.478ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
36072024-11-28T02:03:40.478ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 22 reason = Timeout
36082024-11-28T02:03:40.479ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
36092024-11-28T02:03:40.479ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 22 reason = Timeout
36102024-11-28T02:03:40.479ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
36112024-11-28T02:03:40.479ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 26 reason = Timeout
36122024-11-28T02:03:40.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
36132024-11-28T02:03:40.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
36142024-11-28T02:03:40.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
36152024-11-28T02:03:40.479ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
36162024-11-28T02:03:40.479ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 22 reason = Timeout
36172024-11-28T02:03:40.479ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 22 reason = Timeout
36182024-11-28T02:03:40.479ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
36192024-11-28T02:03:40.479ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
36202024-11-28T02:03:40.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
36212024-11-28T02:03:40.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
36222024-11-28T02:03:40.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
36232024-11-28T02:03:40.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
36242024-11-28T02:03:40.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
36252024-11-28T02:03:40.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
36262024-11-28T02:03:40.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
36272024-11-28T02:03:40.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
36282024-11-28T02:03:40.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
36292024-11-28T02:03:40.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
36302024-11-28T02:03:40.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
36312024-11-28T02:03:40.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
36322024-11-28T02:03:40.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
36332024-11-28T02:03:40.482ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-313847b3-d670-4167-9b09-9e7d3dfcb2a4.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-313847b3-d670-4167-9b09-9e7d3dfcb2a4.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1486, tv_nsec: 952299076 } })
36342024-11-28T02:03:40.482ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1103
36352024-11-28T02:03:40.482ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.99259ms iteration = 22 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
36362024-11-28T02:03:40.483ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
36372024-11-28T02:03:40.483ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
36382024-11-28T02:03:40.484ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
36392024-11-28T02:03:40.484ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
36402024-11-28T02:03:40.485ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
36412024-11-28T02:03:40.485ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
36422024-11-28T02:03:40.485ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
36432024-11-28T02:03:40.485ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-313847b3-d670-4167-9b09-9e7d3dfcb2a4.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-313847b3-d670-4167-9b09-9e7d3dfcb2a4.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1486, tv_nsec: 956074571 } })
36442024-11-28T02:03:40.486ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1103
36452024-11-28T02:03:40.486ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
36462024-11-28T02:03:40.487ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
36472024-11-28T02:03:40.487ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
36482024-11-28T02:03:40.487ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
36492024-11-28T02:03:40.487ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
36502024-11-28T02:03:40.487ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
36512024-11-28T02:03:40.487ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
36522024-11-28T02:03:40.489ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
36532024-11-28T02:03:40.490ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
36542024-11-28T02:03:40.491ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
36552024-11-28T02:03:40.492ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
36562024-11-28T02:03:40.492ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
36572024-11-28T02:03:40.492ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
36582024-11-28T02:03:40.493ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
36592024-11-28T02:03:40.493ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
36602024-11-28T02:03:40.494ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
36612024-11-28T02:03:40.494ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
36622024-11-28T02:03:40.494ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
36632024-11-28T02:03:40.495ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
36642024-11-28T02:03:40.495ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
36652024-11-28T02:03:40.495ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
36662024-11-28T02:03:40.496ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 18.241437ms iteration = 64 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
36672024-11-28T02:03:40.498ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
36682024-11-28T02:03:40.498ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
36692024-11-28T02:03:40.498ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
36702024-11-28T02:03:40.500ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
36712024-11-28T02:03:40.500ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
36722024-11-28T02:03:40.500ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
36732024-11-28T02:03:40.503ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
36742024-11-28T02:03:40.503ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
36752024-11-28T02:03:40.503ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
36762024-11-28T02:03:40.503ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
36772024-11-28T02:03:40.525ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 47.146589ms iteration = 22 status = {"errors":[],"garbage_collect_requested":[]}
36782024-11-28T02:03:40.535ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
36792024-11-28T02:03:40.537ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
36802024-11-28T02:03:40.537ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
36812024-11-28T02:03:40.539ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
36822024-11-28T02:03:40.539ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
36832024-11-28T02:03:40.539ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
36842024-11-28T02:03:40.539ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
36852024-11-28T02:03:40.539ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
36862024-11-28T02:03:40.539ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 60.717615ms iteration = 22 status = {"errors":[],"requests_created_ok":[],"start_invoked_ok":[]}
36872024-11-28T02:03:40.540ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
36882024-11-28T02:03:40.540ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
36892024-11-28T02:03:40.540ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
36902024-11-28T02:03:40.540ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
36912024-11-28T02:03:40.541ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "db2fb3b4-de91-43f9-8f63-382857b22da3", "content-length": "26", "date": "Thu, 28 Nov 2024 02:03:40 GMT"} })
36922024-11-28T02:03:40.541ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
36932024-11-28T02:03:40.542ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
36942024-11-28T02:03:40.542ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
36952024-11-28T02:03:40.542ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
36962024-11-28T02:03:40.542ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
36972024-11-28T02:03:40.543ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-313847b3-d670-4167-9b09-9e7d3dfcb2a4.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-313847b3-d670-4167-9b09-9e7d3dfcb2a4.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1487, tv_nsec: 13446278 } })
36982024-11-28T02:03:40.543ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1103 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc
36992024-11-28T02:03:40.543ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
37002024-11-28T02:03:40.545ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
37012024-11-28T02:03:40.546ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
37022024-11-28T02:03:40.546ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
37032024-11-28T02:03:40.546ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "288299ab-aa21-4a93-9c5f-4d3db5e82686", "content-length": "26", "date": "Thu, 28 Nov 2024 02:03:40 GMT"} })
37042024-11-28T02:03:40.546ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
37052024-11-28T02:03:40.548ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
37062024-11-28T02:03:40.548ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
37072024-11-28T02:03:40.548ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 313847b3-d670-4167-9b09-9e7d3dfcb2a4, lookup_type: ById(313847b3-d670-4167-9b09-9e7d3dfcb2a4) } result = Ok(())
37082024-11-28T02:03:40.548ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
37092024-11-28T02:03:40.548ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
37102024-11-28T02:03:40.549ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
37112024-11-28T02:03:40.550ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
37122024-11-28T02:03:40.550ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
37132024-11-28T02:03:40.550ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
37142024-11-28T02:03:40.550ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
37152024-11-28T02:03:40.551ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 72.215604ms iteration = 22 status = 0
37162024-11-28T02:03:40.552ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 73.516937ms iteration = 22 status = {"errors":[],"records_set_to_done":[]}
37172024-11-28T02:03:40.553ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 74.916251ms iteration = 22 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
37182024-11-28T02:03:40.555ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
37192024-11-28T02:03:40.555ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 76.846114ms iteration = 22 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
37202024-11-28T02:03:40.558ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
37212024-11-28T02:03:40.562ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 67ac247f-c934-4c78-857a-fba4fbeb9937, lookup_type: ById(67ac247f-c934-4c78-857a-fba4fbeb9937) } result = Ok(())
37222024-11-28T02:03:40.562ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
37232024-11-28T02:03:40.562ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
37242024-11-28T02:03:40.565ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
37252024-11-28T02:03:40.568ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
37262024-11-28T02:03:40.568ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
37272024-11-28T02:03:40.569ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
37282024-11-28T02:03:40.579ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 67ac247f-c934-4c78-857a-fba4fbeb9937 original_time_created = 2024-11-28 01:53:03.159475 UTC target_id = 67ac247f-c934-4c78-857a-fba4fbeb9937 time_created = 2024-11-28 01:53:03.159475 UTC
37292024-11-28T02:03:40.579ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 101.372131ms iteration = 64 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"67ac247f-c934-4c78-857a-fba4fbeb9937","time_created":"2024-11-28T01:53:03.159475Z"}
37302024-11-28T02:03:40.583ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 104.978354ms iteration = 22 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[]}
37312024-11-28T02:03:40.598ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
37322024-11-28T02:03:40.598ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
37332024-11-28T02:03:40.598ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 120.048145ms iteration = 22 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
37342024-11-28T02:03:40.628ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
37352024-11-28T02:03:40.629ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "7330be76-72dc-498d-9981-90bce1ed6d6d", "content-length": "1", "date": "Thu, 28 Nov 2024 02:03:40 GMT"} })
37362024-11-28T02:03:40.629ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 150.584426ms iteration = 22 status = {}
37372024-11-28T02:03:40.630ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
37382024-11-28T02:03:40.630ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
37392024-11-28T02:03:40.633ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 154.672896ms iteration = 22 status = null
37402024-11-28T02:03:40.640ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1111 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc zone_address = fd00:1122:3344:101::2
37412024-11-28T02:03:40.640ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
37422024-11-28T02:03:40.640ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "7900110a-81c1-4e0b-a0e2-0b787e49f637", "content-length": "26", "date": "Thu, 28 Nov 2024 02:03:40 GMT"} })
37432024-11-28T02:03:40.640ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1116 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
37442024-11-28T02:03:40.642ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc
37452024-11-28T02:03:40.662ZINFOnexus (BackgroundTasks): VPC route manager sled 313847b3-d670-4167-9b09-9e7d3dfcb2a4 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
37462024-11-28T02:03:40.662ZDEBGnexus (BackgroundTasks): client request SledAgent = 313847b3-d670-4167-9b09-9e7d3dfcb2a4 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
37472024-11-28T02:03:40.662ZDEBGnexus (BackgroundTasks): client response SledAgent = 313847b3-d670-4167-9b09-9e7d3dfcb2a4 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "0a073311-7b31-4b42-b2db-fae93b35e02b", "content-length": "372", "date": "Thu, 28 Nov 2024 02:03:40 GMT"} })
37482024-11-28T02:03:40.662ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
37492024-11-28T02:03:40.662ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
37502024-11-28T02:03:40.665ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
37512024-11-28T02:03:40.666ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
37522024-11-28T02:03:40.668ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
37532024-11-28T02:03:40.670ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
37542024-11-28T02:03:40.670ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
37552024-11-28T02:03:40.670ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
37562024-11-28T02:03:40.672ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
37572024-11-28T02:03:40.673ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 194.141312ms iteration = 26 status = {"physical_disks_added":0}
37582024-11-28T02:03:40.675ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
37592024-11-28T02:03:40.675ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
37602024-11-28T02:03:40.675ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
37612024-11-28T02:03:40.725ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
37622024-11-28T02:03:40.725ZDEBGnexus (BackgroundTasks): client request SledAgent = 313847b3-d670-4167-9b09-9e7d3dfcb2a4 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
37632024-11-28T02:03:40.751ZDEBGnexus (BackgroundTasks): client request SledAgent = 313847b3-d670-4167-9b09-9e7d3dfcb2a4 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
37642024-11-28T02:03:40.789ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
37652024-11-28T02:03:40.790ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
37662024-11-28T02:03:40.792ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 4bb2986f-2e92-4ac3-9dc1-591b6bb60b3e, rack_id: 92a8ff6e-21d3-4c28-bcad-e323847985dc, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(ea300318-9d62-42ed-80ea-21c86590a166) } rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc switch_location = Switch0
37672024-11-28T02:03:40.792ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
37682024-11-28T02:03:40.793ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
37692024-11-28T02:03:40.802ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
37702024-11-28T02:03:40.802ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "c47cb6ab-7ef5-479d-952b-2f1aed39cda5", "content-length": "134", "date": "Thu, 28 Nov 2024 02:03:40 GMT"} })
37712024-11-28T02:03:40.802ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1746 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
37722024-11-28T02:03:40.802ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1773 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = ea300318-9d62-42ed-80ea-21c86590a166
37732024-11-28T02:03:40.802ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
37742024-11-28T02:03:40.802ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
37752024-11-28T02:03:40.802ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
37762024-11-28T02:03:40.802ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
37772024-11-28T02:03:40.802ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
37782024-11-28T02:03:40.802ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1920 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
37792024-11-28T02:03:40.803ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
37802024-11-28T02:03:40.803ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
37812024-11-28T02:03:40.803ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc
37822024-11-28T02:03:40.803ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
37832024-11-28T02:03:40.803ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
37842024-11-28T02:03:40.805ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
37852024-11-28T02:03:40.807ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "0f3e4cbd-6e07-4c18-a1c7-7b80d528c6a3", "content-length": "2", "date": "Thu, 28 Nov 2024 02:03:40 GMT"} })
37862024-11-28T02:03:40.808ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
37872024-11-28T02:03:40.808ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "9b7a213a-005f-47b1-9b6c-ae2ec4260f7d", "content-length": "52", "date": "Thu, 28 Nov 2024 02:03:40 GMT"} })
37882024-11-28T02:03:40.808ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc switch_location = Switch0
37892024-11-28T02:03:40.808ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
37902024-11-28T02:03:40.825ZDEBGnexus (BackgroundTasks): client response SledAgent = 313847b3-d670-4167-9b09-9e7d3dfcb2a4 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "d559d4b6-a0f5-4080-bfe1-265e78e5ab27", "date": "Thu, 28 Nov 2024 02:03:40 GMT"} })
37912024-11-28T02:03:40.825ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
37922024-11-28T02:03:40.825ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
37932024-11-28T02:03:40.827ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
37942024-11-28T02:03:40.827ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
37952024-11-28T02:03:40.830ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
37962024-11-28T02:03:40.830ZDEBGnexus (BackgroundTasks): client response SledAgent = 313847b3-d670-4167-9b09-9e7d3dfcb2a4 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "8a4842bd-08c8-42c0-ab68-0d9c7cf6ab2e", "content-length": "691", "date": "Thu, 28 Nov 2024 02:03:40 GMT"} })
37972024-11-28T02:03:40.830ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
37982024-11-28T02:03:40.830ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
37992024-11-28T02:03:40.830ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
38002024-11-28T02:03:40.830ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
38012024-11-28T02:03:40.832ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 353.550512ms iteration = 22 status = {}
38022024-11-28T02:03:40.834ZINFOnexus (BackgroundTasks): VPC route manager sled 313847b3-d670-4167-9b09-9e7d3dfcb2a4 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
38032024-11-28T02:03:40.834ZDEBGnexus (BackgroundTasks): client request SledAgent = 313847b3-d670-4167-9b09-9e7d3dfcb2a4 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
38042024-11-28T02:03:40.835ZDEBGnexus (BackgroundTasks): client response SledAgent = 313847b3-d670-4167-9b09-9e7d3dfcb2a4 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "eb957fc6-2ddd-4e3c-9df8-f2023833e32a", "date": "Thu, 28 Nov 2024 02:03:40 GMT"} })
38052024-11-28T02:03:40.836ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 358.048229ms iteration = 23 status = {}
38062024-11-28T02:03:40.938ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "473df4f1-fb12-4aa4-815b-b925d3cc42fe", "date": "Thu, 28 Nov 2024 02:03:40 GMT"} })
38072024-11-28T02:03:40.938ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc switch = Switch0
38082024-11-28T02:03:40.938ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
38092024-11-28T02:03:40.939ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "8a72afb8-feae-4f4e-84ee-822481f8a647", "content-length": "682", "date": "Thu, 28 Nov 2024 02:03:40 GMT"} })
38102024-11-28T02:03:40.939ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
38112024-11-28T02:03:40.939ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
38122024-11-28T02:03:40.942ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
38132024-11-28T02:03:40.942ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
38142024-11-28T02:03:40.946ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
38152024-11-28T02:03:40.946ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
38162024-11-28T02:03:40.948ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
38172024-11-28T02:03:40.948ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
38182024-11-28T02:03:40.956ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 477.26303ms iteration = 22 status = {}
38192024-11-28T02:03:41.085ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 54acd377-2b80-4932-9ff0-5d057b8e4d24 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
38202024-11-28T02:03:41.086ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 54acd377-2b80-4932-9ff0-5d057b8e4d24 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
38212024-11-28T02:03:41.088ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2402 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 54acd377-2b80-4932-9ff0-5d057b8e4d24 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
38222024-11-28T02:03:41.380ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:40402 req_id = fe4da4a1-af75-4361-804d-3bb09636cfe2 roles = RoleSet { roles: {} } uri = /nat/ipv4/changeset/6?limit=100
38232024-11-28T02:03:41.381ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:40402 req_id = fe4da4a1-af75-4361-804d-3bb09636cfe2 resource = Database result = Ok(()) uri = /nat/ipv4/changeset/6?limit=100
38242024-11-28T02:03:41.383ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2750 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:40402 req_id = fe4da4a1-af75-4361-804d-3bb09636cfe2 response_code = 200 uri = /nat/ipv4/changeset/6?limit=100
38252024-11-28T02:03:42.105ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 1334d20a-9631-4f8a-9e60-2bab9deaedda roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
38262024-11-28T02:03:42.106ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 1334d20a-9631-4f8a-9e60-2bab9deaedda resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
38272024-11-28T02:03:42.108ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2106 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 1334d20a-9631-4f8a-9e60-2bab9deaedda response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
38282024-11-28T02:03:43.127ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 955fbd9b-afea-4ba4-90d9-aa0e962674ca roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
38292024-11-28T02:03:43.127ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 955fbd9b-afea-4ba4-90d9-aa0e962674ca resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
38302024-11-28T02:03:43.129ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2458 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 955fbd9b-afea-4ba4-90d9-aa0e962674ca response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
38312024-11-28T02:03:44.148ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 50533ab8-cbdf-4609-b198-cc393b842c03 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
38322024-11-28T02:03:44.148ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 50533ab8-cbdf-4609-b198-cc393b842c03 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
38332024-11-28T02:03:44.150ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1900 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 50533ab8-cbdf-4609-b198-cc393b842c03 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
38342024-11-28T02:03:44.195ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1683 local_addr = [fd00:1122:3344:101::b]:55621 method = GET remote_addr = [fd00:1122:3344:101::d]:56417 req_id = 21e95767-b798-4f0e-9540-a4374e9fa48a response_code = 200 uri = /2469a748-bcd2-4e0d-99cc-f79def0452cc
38352024-11-28T02:03:45.169ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 10a16c66-aa29-42e4-a39d-b96f2ab6461e roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
38362024-11-28T02:03:45.169ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 10a16c66-aa29-42e4-a39d-b96f2ab6461e resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
38372024-11-28T02:03:45.171ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1957 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 10a16c66-aa29-42e4-a39d-b96f2ab6461e response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
38382024-11-28T02:03:45.280ZDEBGnexus (producer-registration-task): registering / renewing oximeter producer lease with Nexus
38392024-11-28T02:03:45.280ZDEBGnexus (producer-registration-task): using nexus address for registration addr = [fd00:1122:3344:101::b]:12221
38402024-11-28T02:03:45.310ZDEBGnexus (producer-registration-task): client request body = Some(Body) method = POST uri = http://[fd00:1122:3344:101::b]:12221/metrics/producers
38412024-11-28T02:03:45.310ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::b]:12221 remote_addr = [fd00:1122:3344:101::b]:40720
38422024-11-28T02:03:45.310ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::b]:40720 req_id = 1a41e425-743a-4aab-9982-eaad7e3b633a roles = RoleSet { roles: {} } uri = /metrics/producers
38432024-11-28T02:03:45.311ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::b]:40720 req_id = 1a41e425-743a-4aab-9982-eaad7e3b633a resource = Database result = Ok(()) uri = /metrics/producers
38442024-11-28T02:03:45.318ZINFOnexus: assigned collector to new producer collector_id = 119eed64-2e24-4d5f-8c42-be6cd8bec8b5 file = nexus/src/app/oximeter.rs:76 producer_id = 2469a748-bcd2-4e0d-99cc-f79def0452cc
38452024-11-28T02:03:45.318ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 7446 local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::b]:40720 req_id = 1a41e425-743a-4aab-9982-eaad7e3b633a response_code = 201 uri = /metrics/producers
38462024-11-28T02:03:45.318ZDEBGnexus (producer-registration-task): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::b]:12221/metrics/producers", status: 201, headers: {"content-type": "application/json", "x-request-id": "1a41e425-743a-4aab-9982-eaad7e3b633a", "content-length": "41", "date": "Thu, 28 Nov 2024 02:03:45 GMT"} })
38472024-11-28T02:03:45.319ZDEBGnexus (producer-registration-task): registered with nexus successfully
38482024-11-28T02:03:45.319ZDEBGnexus (producer-registration-task): pausing until time to renew lease lease_duration = 600s wait_period = 150s
38492024-11-28T02:03:46.190ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 5b708f84-4011-45b0-aed7-52a0aab36383 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
38502024-11-28T02:03:46.190ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 5b708f84-4011-45b0-aed7-52a0aab36383 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
38512024-11-28T02:03:46.192ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2332 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 5b708f84-4011-45b0-aed7-52a0aab36383 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
38522024-11-28T02:03:47.212ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = db9b0690-5734-48d8-9492-6f948fafffe4 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
38532024-11-28T02:03:47.213ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = db9b0690-5734-48d8-9492-6f948fafffe4 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
38542024-11-28T02:03:47.214ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2302 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = db9b0690-5734-48d8-9492-6f948fafffe4 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
38552024-11-28T02:03:48.234ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 8fd8ab0e-4921-47a3-9225-c6b4bba474f8 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
38562024-11-28T02:03:48.235ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 8fd8ab0e-4921-47a3-9225-c6b4bba474f8 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
38572024-11-28T02:03:48.237ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2343 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 8fd8ab0e-4921-47a3-9225-c6b4bba474f8 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
38582024-11-28T02:03:49.257ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 3ebb563e-68da-4ea4-8a84-823d768f683f roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
38592024-11-28T02:03:49.257ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 3ebb563e-68da-4ea4-8a84-823d768f683f resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
38602024-11-28T02:03:49.259ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1854 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 3ebb563e-68da-4ea4-8a84-823d768f683f response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
38612024-11-28T02:03:50.277ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 712603ec-6a71-4652-9b82-49077e3c38c2 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
38622024-11-28T02:03:50.277ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 712603ec-6a71-4652-9b82-49077e3c38c2 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
38632024-11-28T02:03:50.278ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1984 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 712603ec-6a71-4652-9b82-49077e3c38c2 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
38642024-11-28T02:03:50.479ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 65 reason = Timeout
38652024-11-28T02:03:50.479ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 65 reason = Timeout
38662024-11-28T02:03:50.479ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
38672024-11-28T02:03:50.479ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
38682024-11-28T02:03:50.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
38692024-11-28T02:03:50.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
38702024-11-28T02:03:50.480ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
38712024-11-28T02:03:50.481ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
38722024-11-28T02:03:50.481ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
38732024-11-28T02:03:50.483ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 4.061021ms iteration = 65 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
38742024-11-28T02:03:50.484ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
38752024-11-28T02:03:50.484ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
38762024-11-28T02:03:50.484ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
38772024-11-28T02:03:50.485ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
38782024-11-28T02:03:50.485ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
38792024-11-28T02:03:50.487ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
38802024-11-28T02:03:50.489ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 67ac247f-c934-4c78-857a-fba4fbeb9937, lookup_type: ById(67ac247f-c934-4c78-857a-fba4fbeb9937) } result = Ok(())
38812024-11-28T02:03:50.489ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
38822024-11-28T02:03:50.489ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
38832024-11-28T02:03:50.502ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 67ac247f-c934-4c78-857a-fba4fbeb9937 original_time_created = 2024-11-28 01:53:03.159475 UTC target_id = 67ac247f-c934-4c78-857a-fba4fbeb9937 time_created = 2024-11-28 01:53:03.159475 UTC
38842024-11-28T02:03:50.502ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 23.372366ms iteration = 65 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"67ac247f-c934-4c78-857a-fba4fbeb9937","time_created":"2024-11-28T01:53:03.159475Z"}
38852024-11-28T02:03:51.297ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 42bf9b8e-5570-47ff-8104-a2b3d09dc334 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
38862024-11-28T02:03:51.298ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 42bf9b8e-5570-47ff-8104-a2b3d09dc334 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
38872024-11-28T02:03:51.300ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2204 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 42bf9b8e-5570-47ff-8104-a2b3d09dc334 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
38882024-11-28T02:03:52.318ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = aad4dd4e-24d7-469b-98b1-77ed5a1ae761 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
38892024-11-28T02:03:52.318ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = aad4dd4e-24d7-469b-98b1-77ed5a1ae761 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
38902024-11-28T02:03:52.320ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2195 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = aad4dd4e-24d7-469b-98b1-77ed5a1ae761 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
38912024-11-28T02:03:53.338ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 504db85b-e112-4e47-83c3-30b577e9d1e2 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
38922024-11-28T02:03:53.338ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 504db85b-e112-4e47-83c3-30b577e9d1e2 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
38932024-11-28T02:03:53.340ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2769 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 504db85b-e112-4e47-83c3-30b577e9d1e2 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
38942024-11-28T02:03:54.186ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:55495 req_id = 63e97509-dfb5-43a7-b720-4b00c862d39d roles = RoleSet { roles: {} } uri = /metrics/collectors/119eed64-2e24-4d5f-8c42-be6cd8bec8b5/producers?sort_by=id_ascending
38952024-11-28T02:03:54.186ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:55495 req_id = 63e97509-dfb5-43a7-b720-4b00c862d39d resource = Database result = Ok(()) uri = /metrics/collectors/119eed64-2e24-4d5f-8c42-be6cd8bec8b5/producers?sort_by=id_ascending
38962024-11-28T02:03:54.188ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2341 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:55495 req_id = 63e97509-dfb5-43a7-b720-4b00c862d39d response_code = 200 uri = /metrics/collectors/119eed64-2e24-4d5f-8c42-be6cd8bec8b5/producers?sort_by=id_ascending
38972024-11-28T02:03:54.188ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:55495 req_id = ae62860b-00de-4c6b-bb24-3504bad0e2ee roles = RoleSet { roles: {} } uri = /metrics/collectors/119eed64-2e24-4d5f-8c42-be6cd8bec8b5/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJjNDM0YjQ0OC01MTk5LTQ2NmUtYWYzMi00ZjYyOWUzMWM3YjEifX0%3D
38982024-11-28T02:03:54.188ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:55495 req_id = ae62860b-00de-4c6b-bb24-3504bad0e2ee resource = Database result = Ok(()) uri = /metrics/collectors/119eed64-2e24-4d5f-8c42-be6cd8bec8b5/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJjNDM0YjQ0OC01MTk5LTQ2NmUtYWYzMi00ZjYyOWUzMWM3YjEifX0%3D
38992024-11-28T02:03:54.190ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2089 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:55495 req_id = ae62860b-00de-4c6b-bb24-3504bad0e2ee response_code = 200 uri = /metrics/collectors/119eed64-2e24-4d5f-8c42-be6cd8bec8b5/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJjNDM0YjQ0OC01MTk5LTQ2NmUtYWYzMi00ZjYyOWUzMWM3YjEifX0%3D
39002024-11-28T02:03:54.194ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1218 local_addr = [fd00:1122:3344:101::b]:55621 method = GET remote_addr = [fd00:1122:3344:101::d]:56417 req_id = 6e8d74dc-4f0a-4b58-aeb7-4fc6d4c996ee response_code = 200 uri = /2469a748-bcd2-4e0d-99cc-f79def0452cc
39012024-11-28T02:03:54.359ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = f3f0b3af-22c1-4eef-a10d-63ea07b27f24 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
39022024-11-28T02:03:54.359ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = f3f0b3af-22c1-4eef-a10d-63ea07b27f24 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
39032024-11-28T02:03:54.360ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1800 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = f3f0b3af-22c1-4eef-a10d-63ea07b27f24 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
39042024-11-28T02:03:55.380ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 5446260d-283d-481d-8bf2-57e171d9ed0b roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
39052024-11-28T02:03:55.381ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 5446260d-283d-481d-8bf2-57e171d9ed0b resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
39062024-11-28T02:03:55.382ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2315 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 5446260d-283d-481d-8bf2-57e171d9ed0b response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
39072024-11-28T02:03:56.401ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 5d349354-5057-45b2-a828-eea6d2775524 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
39082024-11-28T02:03:56.402ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 5d349354-5057-45b2-a828-eea6d2775524 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
39092024-11-28T02:03:56.403ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2138 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 5d349354-5057-45b2-a828-eea6d2775524 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
39102024-11-28T02:03:57.423ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = e8fb2564-7ac9-4fb6-8594-f9aa9a441b21 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
39112024-11-28T02:03:57.423ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = e8fb2564-7ac9-4fb6-8594-f9aa9a441b21 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
39122024-11-28T02:03:57.424ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1804 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = e8fb2564-7ac9-4fb6-8594-f9aa9a441b21 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
39132024-11-28T02:03:58.444ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 62f7a310-b0a4-4bb4-88b8-5d68bcd70bd6 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
39142024-11-28T02:03:58.444ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 62f7a310-b0a4-4bb4-88b8-5d68bcd70bd6 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
39152024-11-28T02:03:58.445ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1695 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 62f7a310-b0a4-4bb4-88b8-5d68bcd70bd6 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
39162024-11-28T02:03:59.465ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = d716da08-fc3d-4deb-920b-eda10525c774 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
39172024-11-28T02:03:59.466ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = d716da08-fc3d-4deb-920b-eda10525c774 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
39182024-11-28T02:03:59.467ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2099 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = d716da08-fc3d-4deb-920b-eda10525c774 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
39192024-11-28T02:04:00.478ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 66 reason = Timeout
39202024-11-28T02:04:00.478ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 66 reason = Timeout
39212024-11-28T02:04:00.478ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
39222024-11-28T02:04:00.478ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
39232024-11-28T02:04:00.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
39242024-11-28T02:04:00.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
39252024-11-28T02:04:00.480ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
39262024-11-28T02:04:00.481ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
39272024-11-28T02:04:00.481ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
39282024-11-28T02:04:00.482ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.805926ms iteration = 66 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
39292024-11-28T02:04:00.483ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
39302024-11-28T02:04:00.483ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
39312024-11-28T02:04:00.483ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
39322024-11-28T02:04:00.485ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
39332024-11-28T02:04:00.486ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
39342024-11-28T02:04:00.487ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = df770cb2-4468-4a3e-afe8-c37f935304d8 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
39352024-11-28T02:04:00.487ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = df770cb2-4468-4a3e-afe8-c37f935304d8 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
39362024-11-28T02:04:00.487ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
39372024-11-28T02:04:00.488ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1619 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = df770cb2-4468-4a3e-afe8-c37f935304d8 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
39382024-11-28T02:04:00.490ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 67ac247f-c934-4c78-857a-fba4fbeb9937, lookup_type: ById(67ac247f-c934-4c78-857a-fba4fbeb9937) } result = Ok(())
39392024-11-28T02:04:00.490ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
39402024-11-28T02:04:00.490ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
39412024-11-28T02:04:00.499ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 67ac247f-c934-4c78-857a-fba4fbeb9937 original_time_created = 2024-11-28 01:53:03.159475 UTC target_id = 67ac247f-c934-4c78-857a-fba4fbeb9937 time_created = 2024-11-28 01:53:03.159475 UTC
39422024-11-28T02:04:00.499ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 20.493336ms iteration = 66 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"67ac247f-c934-4c78-857a-fba4fbeb9937","time_created":"2024-11-28T01:53:03.159475Z"}
39432024-11-28T02:04:01.508ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = c275197b-710d-47d4-84cb-1cb959595520 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
39442024-11-28T02:04:01.509ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = c275197b-710d-47d4-84cb-1cb959595520 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
39452024-11-28T02:04:01.513ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 4538 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = c275197b-710d-47d4-84cb-1cb959595520 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
39462024-11-28T02:04:02.537ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 16fae8d9-4f01-4d27-9760-b87d4deae76c roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
39472024-11-28T02:04:02.538ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 16fae8d9-4f01-4d27-9760-b87d4deae76c resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
39482024-11-28T02:04:02.540ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2557 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 16fae8d9-4f01-4d27-9760-b87d4deae76c response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
39492024-11-28T02:04:03.568ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 0528038f-b06b-4ff3-9c59-8d2665a578ea roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
39502024-11-28T02:04:03.568ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 0528038f-b06b-4ff3-9c59-8d2665a578ea resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
39512024-11-28T02:04:03.571ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 3184 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 0528038f-b06b-4ff3-9c59-8d2665a578ea response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
39522024-11-28T02:04:04.194ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1403 local_addr = [fd00:1122:3344:101::b]:55621 method = GET remote_addr = [fd00:1122:3344:101::d]:56417 req_id = 29dd48d2-98bb-40a6-97e6-c7303deeabd2 response_code = 200 uri = /2469a748-bcd2-4e0d-99cc-f79def0452cc
39532024-11-28T02:04:04.599ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = d5f1f49a-8f53-4a7a-8c8c-c76708c99f8c roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
39542024-11-28T02:04:04.600ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = d5f1f49a-8f53-4a7a-8c8c-c76708c99f8c resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
39552024-11-28T02:04:04.601ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1983 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = d5f1f49a-8f53-4a7a-8c8c-c76708c99f8c response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
39562024-11-28T02:04:05.620ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = f17011da-de0b-4255-8e75-54a0f8e16bd6 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
39572024-11-28T02:04:05.620ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = f17011da-de0b-4255-8e75-54a0f8e16bd6 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
39582024-11-28T02:04:05.622ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2561 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = f17011da-de0b-4255-8e75-54a0f8e16bd6 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
39592024-11-28T02:04:06.640ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 0f75ea34-fc4f-44c2-9133-3a8fcfe7e2d3 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
39602024-11-28T02:04:06.640ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 0f75ea34-fc4f-44c2-9133-3a8fcfe7e2d3 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
39612024-11-28T02:04:06.642ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2279 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 0f75ea34-fc4f-44c2-9133-3a8fcfe7e2d3 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
39622024-11-28T02:04:07.661ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = a3b45785-a9c5-4de9-a175-e346957cdf36 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
39632024-11-28T02:04:07.661ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = a3b45785-a9c5-4de9-a175-e346957cdf36 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
39642024-11-28T02:04:07.663ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1821 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = a3b45785-a9c5-4de9-a175-e346957cdf36 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
39652024-11-28T02:04:08.684ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = b375cb59-cbff-4b9a-a2cc-16aaef986114 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
39662024-11-28T02:04:08.685ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = b375cb59-cbff-4b9a-a2cc-16aaef986114 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
39672024-11-28T02:04:08.687ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2270 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = b375cb59-cbff-4b9a-a2cc-16aaef986114 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
39682024-11-28T02:04:09.706ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 150f5585-48b8-43a3-a207-c59b66fb7c26 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
39692024-11-28T02:04:09.706ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 150f5585-48b8-43a3-a207-c59b66fb7c26 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
39702024-11-28T02:04:09.708ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2091 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 150f5585-48b8-43a3-a207-c59b66fb7c26 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
39712024-11-28T02:04:10.479ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 14 reason = Timeout
39722024-11-28T02:04:10.479ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 23 reason = Timeout
39732024-11-28T02:04:10.479ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 13 reason = Timeout
39742024-11-28T02:04:10.479ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
39752024-11-28T02:04:10.479ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
39762024-11-28T02:04:10.479ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
39772024-11-28T02:04:10.479ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 23 reason = Timeout
39782024-11-28T02:04:10.479ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
39792024-11-28T02:04:10.479ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 13 reason = Timeout
39802024-11-28T02:04:10.479ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
39812024-11-28T02:04:10.479ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
39822024-11-28T02:04:10.479ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 12 reason = Timeout
39832024-11-28T02:04:10.479ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2024-11-28 01:54:10.479420810 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
39842024-11-28T02:04:10.479ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 23 reason = Timeout
39852024-11-28T02:04:10.479ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
39862024-11-28T02:04:10.479ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 23 reason = Timeout
39872024-11-28T02:04:10.479ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
39882024-11-28T02:04:10.479ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
39892024-11-28T02:04:10.479ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
39902024-11-28T02:04:10.479ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 14 reason = Timeout
39912024-11-28T02:04:10.479ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 67 reason = Timeout
39922024-11-28T02:04:10.479ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
39932024-11-28T02:04:10.479ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 23 reason = Timeout
39942024-11-28T02:04:10.479ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
39952024-11-28T02:04:10.479ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 23 reason = Timeout
39962024-11-28T02:04:10.479ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
39972024-11-28T02:04:10.479ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 23 reason = Timeout
39982024-11-28T02:04:10.479ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
39992024-11-28T02:04:10.479ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 23 reason = Timeout
40002024-11-28T02:04:10.479ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 23 reason = Timeout
40012024-11-28T02:04:10.479ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
40022024-11-28T02:04:10.479ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
40032024-11-28T02:04:10.479ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 27 reason = Timeout
40042024-11-28T02:04:10.479ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
40052024-11-28T02:04:10.479ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 23 reason = Timeout
40062024-11-28T02:04:10.479ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
40072024-11-28T02:04:10.479ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 23 reason = Timeout
40082024-11-28T02:04:10.479ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 12 reason = Timeout
40092024-11-28T02:04:10.479ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
40102024-11-28T02:04:10.479ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 12 reason = Timeout
40112024-11-28T02:04:10.479ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 24 reason = Timeout
40122024-11-28T02:04:10.479ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
40132024-11-28T02:04:10.479ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
40142024-11-28T02:04:10.479ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 23 reason = Timeout
40152024-11-28T02:04:10.479ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
40162024-11-28T02:04:10.479ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
40172024-11-28T02:04:10.479ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 13 reason = Timeout
40182024-11-28T02:04:10.479ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 23 reason = Timeout
40192024-11-28T02:04:10.479ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
40202024-11-28T02:04:10.479ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 13 reason = Timeout
40212024-11-28T02:04:10.479ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 13 reason = Timeout
40222024-11-28T02:04:10.479ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:83 reason = blueprint disabled target_id = 67ac247f-c934-4c78-857a-fba4fbeb9937
40232024-11-28T02:04:10.479ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 67 reason = Timeout
40242024-11-28T02:04:10.479ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 15.64\u{b5}s iteration = 13 status = {"enabled":false,"target_id":"67ac247f-c934-4c78-857a-fba4fbeb9937"}
40252024-11-28T02:04:10.479ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
40262024-11-28T02:04:10.479ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 12 reason = Timeout
40272024-11-28T02:04:10.479ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 12 reason = Timeout
40282024-11-28T02:04:10.479ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:56
40292024-11-28T02:04:10.479ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
40302024-11-28T02:04:10.479ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
40312024-11-28T02:04:10.479ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 13 reason = Timeout
40322024-11-28T02:04:10.479ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
40332024-11-28T02:04:10.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
40342024-11-28T02:04:10.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
40352024-11-28T02:04:10.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
40362024-11-28T02:04:10.480ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
40372024-11-28T02:04:10.480ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
40382024-11-28T02:04:10.480ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
40392024-11-28T02:04:10.480ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
40402024-11-28T02:04:10.480ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
40412024-11-28T02:04:10.480ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
40422024-11-28T02:04:10.480ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
40432024-11-28T02:04:10.480ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
40442024-11-28T02:04:10.480ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
40452024-11-28T02:04:10.480ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
40462024-11-28T02:04:10.480ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
40472024-11-28T02:04:10.480ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
40482024-11-28T02:04:10.480ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
40492024-11-28T02:04:10.480ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
40502024-11-28T02:04:10.480ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
40512024-11-28T02:04:10.480ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
40522024-11-28T02:04:10.480ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
40532024-11-28T02:04:10.480ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
40542024-11-28T02:04:10.480ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
40552024-11-28T02:04:10.480ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
40562024-11-28T02:04:10.480ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
40572024-11-28T02:04:10.480ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("36d4f615-7461-4364-960c-89ece3aaabaf.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("eb274350-c0b0-4bce-9c8a-502880f2c905.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("eb274350-c0b0-4bce-9c8a-502880f2c905.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::8))) }], valid_until: Instant { tv_sec: 1516, tv_nsec: 950639427 } })
40582024-11-28T02:04:10.480ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("41ea4a99-3038-474e-bf28-2ad338f3243f.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("9402a51b-d30f-4544-93ec-0f5dc7a485e4.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("d23bc5f2-6b92-4992-bdd2-8695e5bf9c0f.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("d23bc5f2-6b92-4992-bdd2-8695e5bf9c0f.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:1::1))) }], valid_until: Instant { tv_sec: 1516, tv_nsec: 950681367 } })
40592024-11-28T02:04:10.481ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } dns_group = external
40602024-11-28T02:04:10.481ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.309133ms iteration = 13 status = {"addresses":["[fd00:1122:3344:101::9]:5353","[fd00:1122:3344:101::8]:5353"]}
40612024-11-28T02:04:10.481ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } dns_group = internal
40622024-11-28T02:04:10.481ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.412774ms iteration = 13 status = {"addresses":["[fd00:1122:3344:3::1]:5353","[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:1::1]:5353"]}
40632024-11-28T02:04:10.481ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
40642024-11-28T02:04:10.481ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
40652024-11-28T02:04:10.482ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-313847b3-d670-4167-9b09-9e7d3dfcb2a4.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-313847b3-d670-4167-9b09-9e7d3dfcb2a4.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1516, tv_nsec: 952861345 } })
40662024-11-28T02:04:10.483ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1103
40672024-11-28T02:04:10.484ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
40682024-11-28T02:04:10.484ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
40692024-11-28T02:04:10.484ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 4.652531ms iteration = 23 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
40702024-11-28T02:04:10.490ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-313847b3-d670-4167-9b09-9e7d3dfcb2a4.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-313847b3-d670-4167-9b09-9e7d3dfcb2a4.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1516, tv_nsec: 960151522 } })
40712024-11-28T02:04:10.490ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1103
40722024-11-28T02:04:10.491ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
40732024-11-28T02:04:10.491ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
40742024-11-28T02:04:10.491ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
40752024-11-28T02:04:10.491ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
40762024-11-28T02:04:10.491ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
40772024-11-28T02:04:10.495ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2024-11-28 01:54:10.479420810 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
40782024-11-28T02:04:10.495ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 16.387065ms iteration = 12 status = {"expiration":"2024-11-28T01:54:10.479420810Z","pruned":[]}
40792024-11-28T02:04:10.496ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
40802024-11-28T02:04:10.496ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
40812024-11-28T02:04:10.496ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
40822024-11-28T02:04:10.499ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 19.692012ms iteration = 12 status = {"errors":[],"found_port_ok":[]}
40832024-11-28T02:04:10.500ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
40842024-11-28T02:04:10.500ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
40852024-11-28T02:04:10.547ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
40862024-11-28T02:04:10.548ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
40872024-11-28T02:04:10.549ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
40882024-11-28T02:04:10.551ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
40892024-11-28T02:04:10.551ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
40902024-11-28T02:04:10.551ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
40912024-11-28T02:04:10.551ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
40922024-11-28T02:04:10.551ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
40932024-11-28T02:04:10.552ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
40942024-11-28T02:04:10.552ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
40952024-11-28T02:04:10.553ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
40962024-11-28T02:04:10.553ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
40972024-11-28T02:04:10.553ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
40982024-11-28T02:04:10.553ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
40992024-11-28T02:04:10.554ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 74.26442ms iteration = 67 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
41002024-11-28T02:04:10.554ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 74.584896ms iteration = 23 status = {"errors":[],"garbage_collect_requested":[]}
41012024-11-28T02:04:10.556ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
41022024-11-28T02:04:10.556ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
41032024-11-28T02:04:10.556ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
41042024-11-28T02:04:10.558ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
41052024-11-28T02:04:10.558ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
41062024-11-28T02:04:10.558ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
41072024-11-28T02:04:10.558ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
41082024-11-28T02:04:10.558ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:171 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
41092024-11-28T02:04:10.558ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 78.993922ms iteration = 12 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
41102024-11-28T02:04:10.558ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
41112024-11-28T02:04:10.560ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
41122024-11-28T02:04:10.560ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
41132024-11-28T02:04:10.560ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
41142024-11-28T02:04:10.560ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
41152024-11-28T02:04:10.561ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
41162024-11-28T02:04:10.561ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 81.419415ms iteration = 23 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
41172024-11-28T02:04:10.561ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
41182024-11-28T02:04:10.561ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
41192024-11-28T02:04:10.561ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
41202024-11-28T02:04:10.561ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
41212024-11-28T02:04:10.562ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
41222024-11-28T02:04:10.562ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
41232024-11-28T02:04:10.563ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 83.389258ms iteration = 23 status = {"errors":[],"records_set_to_done":[]}
41242024-11-28T02:04:10.563ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
41252024-11-28T02:04:10.565ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
41262024-11-28T02:04:10.565ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
41272024-11-28T02:04:10.565ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
41282024-11-28T02:04:10.566ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
41292024-11-28T02:04:10.566ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
41302024-11-28T02:04:10.566ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
41312024-11-28T02:04:10.566ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
41322024-11-28T02:04:10.566ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
41332024-11-28T02:04:10.566ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
41342024-11-28T02:04:10.566ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
41352024-11-28T02:04:10.566ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
41362024-11-28T02:04:10.567ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
41372024-11-28T02:04:10.568ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
41382024-11-28T02:04:10.568ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
41392024-11-28T02:04:10.568ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
41402024-11-28T02:04:10.568ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
41412024-11-28T02:04:10.569ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
41422024-11-28T02:04:10.569ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
41432024-11-28T02:04:10.569ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
41442024-11-28T02:04:10.570ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
41452024-11-28T02:04:10.572ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-313847b3-d670-4167-9b09-9e7d3dfcb2a4.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-313847b3-d670-4167-9b09-9e7d3dfcb2a4.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1517, tv_nsec: 42915549 } })
41462024-11-28T02:04:10.572ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
41472024-11-28T02:04:10.573ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1103 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc
41482024-11-28T02:04:10.573ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
41492024-11-28T02:04:10.573ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
41502024-11-28T02:04:10.574ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
41512024-11-28T02:04:10.574ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
41522024-11-28T02:04:10.575ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
41532024-11-28T02:04:10.575ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
41542024-11-28T02:04:10.575ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
41552024-11-28T02:04:10.578ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
41562024-11-28T02:04:10.578ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
41572024-11-28T02:04:10.578ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
41582024-11-28T02:04:10.578ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 98.910028ms iteration = 23 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
41592024-11-28T02:04:10.578ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
41602024-11-28T02:04:10.578ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
41612024-11-28T02:04:10.578ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
41622024-11-28T02:04:10.578ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
41632024-11-28T02:04:10.581ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
41642024-11-28T02:04:10.581ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
41652024-11-28T02:04:10.583ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
41662024-11-28T02:04:10.584ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
41672024-11-28T02:04:10.585ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
41682024-11-28T02:04:10.588ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 313847b3-d670-4167-9b09-9e7d3dfcb2a4, lookup_type: ById(313847b3-d670-4167-9b09-9e7d3dfcb2a4) } result = Ok(())
41692024-11-28T02:04:10.588ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
41702024-11-28T02:04:10.588ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
41712024-11-28T02:04:10.590ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
41722024-11-28T02:04:10.590ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
41732024-11-28T02:04:10.590ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 111.147301ms iteration = 23 status = 0
41742024-11-28T02:04:10.600ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
41752024-11-28T02:04:10.600ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
41762024-11-28T02:04:10.600ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
41772024-11-28T02:04:10.600ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
41782024-11-28T02:04:10.600ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
41792024-11-28T02:04:10.602ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
41802024-11-28T02:04:10.606ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
41812024-11-28T02:04:10.607ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
41822024-11-28T02:04:10.607ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
41832024-11-28T02:04:10.607ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
41842024-11-28T02:04:10.611ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
41852024-11-28T02:04:10.612ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
41862024-11-28T02:04:10.612ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = a7e6af61-408c-483d-b043-c27ccf55708e dns_zone_name = oxide.test found_so_far = 0 version = 2
41872024-11-28T02:04:10.612ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
41882024-11-28T02:04:10.612ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
41892024-11-28T02:04:10.612ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
41902024-11-28T02:04:10.612ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
41912024-11-28T02:04:10.614ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
41922024-11-28T02:04:10.614ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
41932024-11-28T02:04:10.616ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 67ac247f-c934-4c78-857a-fba4fbeb9937, lookup_type: ById(67ac247f-c934-4c78-857a-fba4fbeb9937) } result = Ok(())
41942024-11-28T02:04:10.616ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
41952024-11-28T02:04:10.616ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
41962024-11-28T02:04:10.618ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
41972024-11-28T02:04:10.618ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
41982024-11-28T02:04:10.618ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
41992024-11-28T02:04:10.619ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
42002024-11-28T02:04:10.619ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 140.132314ms iteration = 23 status = {"errors":[],"requests_created_ok":[],"start_invoked_ok":[]}
42012024-11-28T02:04:10.619ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
42022024-11-28T02:04:10.619ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
42032024-11-28T02:04:10.620ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
42042024-11-28T02:04:10.621ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
42052024-11-28T02:04:10.621ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:140 found = 0 not_ready_to_be_deleted = 0
42062024-11-28T02:04:10.621ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 141.233533ms iteration = 12 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
42072024-11-28T02:04:10.621ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
42082024-11-28T02:04:10.621ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
42092024-11-28T02:04:10.621ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
42102024-11-28T02:04:10.621ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 142.040587ms iteration = 23 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
42112024-11-28T02:04:10.621ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
42122024-11-28T02:04:10.624ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = a7e6af61-408c-483d-b043-c27ccf55708e dns_zone_name = oxide.test found_so_far = 1 version = 2
42132024-11-28T02:04:10.624ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
42142024-11-28T02:04:10.624ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2024-11-28 01:53:43.351579 UTC dns_group = external generation = 2
42152024-11-28T02:04:10.624ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 145.177941ms iteration = 13 status = {"generation":2}
42162024-11-28T02:04:10.625ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
42172024-11-28T02:04:10.625ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
42182024-11-28T02:04:10.625ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
42192024-11-28T02:04:10.625ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "ef9298fb-1b00-4064-9040-a07f40203267", "content-length": "26", "date": "Thu, 28 Nov 2024 02:04:10 GMT"} })
42202024-11-28T02:04:10.625ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
42212024-11-28T02:04:10.626ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
42222024-11-28T02:04:10.626ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
42232024-11-28T02:04:10.626ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "5e33409e-f747-46f4-b31d-e7a2a53a5e97", "content-length": "26", "date": "Thu, 28 Nov 2024 02:04:10 GMT"} })
42242024-11-28T02:04:10.626ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
42252024-11-28T02:04:10.627ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
42262024-11-28T02:04:10.627ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 808a324e-c913-4bfc-91a5-d43c961cd511 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
42272024-11-28T02:04:10.627ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
42282024-11-28T02:04:10.627ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
42292024-11-28T02:04:10.628ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
42302024-11-28T02:04:10.628ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
42312024-11-28T02:04:10.628ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
42322024-11-28T02:04:10.631ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
42332024-11-28T02:04:10.633ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
42342024-11-28T02:04:10.633ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
42352024-11-28T02:04:10.636ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
42362024-11-28T02:04:10.636ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
42372024-11-28T02:04:10.636ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
42382024-11-28T02:04:10.638ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 67ac247f-c934-4c78-857a-fba4fbeb9937 original_time_created = 2024-11-28 01:53:03.159475 UTC target_id = 67ac247f-c934-4c78-857a-fba4fbeb9937 time_created = 2024-11-28 01:53:03.159475 UTC
42392024-11-28T02:04:10.638ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 159.438239ms iteration = 67 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"67ac247f-c934-4c78-857a-fba4fbeb9937","time_created":"2024-11-28T01:53:03.159475Z"}
42402024-11-28T02:04:10.639ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 808a324e-c913-4bfc-91a5-d43c961cd511 dns_zone_name = control-plane.oxide.internal found_so_far = 51 version = 1
42412024-11-28T02:04:10.639ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
42422024-11-28T02:04:10.639ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2024-11-28 01:53:42.502562 UTC dns_group = internal generation = 1
42432024-11-28T02:04:10.639ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 160.315465ms iteration = 13 status = {"generation":1}
42442024-11-28T02:04:10.641ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 161.990713ms iteration = 23 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[]}
42452024-11-28T02:04:10.654ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
42462024-11-28T02:04:10.655ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
42472024-11-28T02:04:10.670ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2024-11-28T01:53:08.890632Z, time_modified: 2024-11-28T01:53:08.890632Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 7e159834-997a-4c03-b9db-1e91d783221c, db_silo: Silo { identity: SiloIdentity { id: 7e159834-997a-4c03-b9db-1e91d783221c, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-11-28T01:53:43.308696Z, time_modified: 2024-11-28T01:53:43.308696Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "bc39d1b3b4bb84a49d6458ed7aedc5bca9e77742c7b35882b64d188c0602d306" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 7e159834-997a-4c03-b9db-1e91d783221c, db_silo: Silo { identity: SiloIdentity { id: 7e159834-997a-4c03-b9db-1e91d783221c, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-11-28T01:53:43.308696Z, time_modified: 2024-11-28T01:53:43.308696Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "bc39d1b3b4bb84a49d6458ed7aedc5bca9e77742c7b35882b64d188c0602d306" }] }) }
42482024-11-28T02:04:10.670ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 190.345467ms iteration = 13 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"7e159834-997a-4c03-b9db-1e91d783221c","tls_certs":["bc39d1b3b4bb84a49d6458ed7aedc5bca9e77742c7b35882b64d188c0602d306"]}},"default_endpoint":{"silo_id":"7e159834-997a-4c03-b9db-1e91d783221c","tls_certs":["bc39d1b3b4bb84a49d6458ed7aedc5bca9e77742c7b35882b64d188c0602d306"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
42492024-11-28T02:04:10.697ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 217.430766ms iteration = 27 status = {"physical_disks_added":0}
42502024-11-28T02:04:10.698ZWARNnexus (BackgroundTasks): instance reincarnation completed with errors background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:103 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0 query_errors = 0 restart_errors = 0
42512024-11-28T02:04:10.698ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 218.673718ms iteration = 12 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
42522024-11-28T02:04:10.699ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
42532024-11-28T02:04:10.699ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
42542024-11-28T02:04:10.701ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
42552024-11-28T02:04:10.703ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
42562024-11-28T02:04:10.703ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
42572024-11-28T02:04:10.703ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
42582024-11-28T02:04:10.728ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = dfc36bb8-7473-49a8-9173-0a9bafaab399 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
42592024-11-28T02:04:10.728ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = dfc36bb8-7473-49a8-9173-0a9bafaab399 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
42602024-11-28T02:04:10.747ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1111 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc zone_address = fd00:1122:3344:101::2
42612024-11-28T02:04:10.747ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
42622024-11-28T02:04:10.748ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "8ea6e761-50a6-419e-be5c-901955dbed1f", "content-length": "26", "date": "Thu, 28 Nov 2024 02:04:10 GMT"} })
42632024-11-28T02:04:10.748ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1116 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
42642024-11-28T02:04:10.750ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc
42652024-11-28T02:04:10.778ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
42662024-11-28T02:04:10.779ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "6507fa90-84d3-4a18-98c8-3315b8513d57", "date": "Thu, 28 Nov 2024 02:04:10 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
42672024-11-28T02:04:10.779ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
42682024-11-28T02:04:10.781ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "e21b6977-c697-47fa-8494-3258f404d72e", "date": "Thu, 28 Nov 2024 02:04:10 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
42692024-11-28T02:04:10.781ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
42702024-11-28T02:04:10.782ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
42712024-11-28T02:04:10.782ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 303.570443ms iteration = 14 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
42722024-11-28T02:04:10.830ZINFOnexus (BackgroundTasks): VPC route manager sled 313847b3-d670-4167-9b09-9e7d3dfcb2a4 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
42732024-11-28T02:04:10.830ZDEBGnexus (BackgroundTasks): client request SledAgent = 313847b3-d670-4167-9b09-9e7d3dfcb2a4 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
42742024-11-28T02:04:10.830ZDEBGnexus (BackgroundTasks): client response SledAgent = 313847b3-d670-4167-9b09-9e7d3dfcb2a4 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "099a0459-caf3-4ed6-9fc6-adb1a61f8c53", "content-length": "372", "date": "Thu, 28 Nov 2024 02:04:10 GMT"} })
42752024-11-28T02:04:10.830ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
42762024-11-28T02:04:10.831ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
42772024-11-28T02:04:10.832ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
42782024-11-28T02:04:10.832ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 104547 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = dfc36bb8-7473-49a8-9173-0a9bafaab399 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
42792024-11-28T02:04:10.834ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
42802024-11-28T02:04:10.834ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
42812024-11-28T02:04:10.834ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
42822024-11-28T02:04:10.879ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
42832024-11-28T02:04:10.879ZDEBGnexus (BackgroundTasks): client request SledAgent = 313847b3-d670-4167-9b09-9e7d3dfcb2a4 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
42842024-11-28T02:04:10.879ZDEBGnexus (BackgroundTasks): client response SledAgent = 313847b3-d670-4167-9b09-9e7d3dfcb2a4 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "110aeab4-0f20-46d2-9325-907cbe9d1a50", "date": "Thu, 28 Nov 2024 02:04:10 GMT"} })
42852024-11-28T02:04:10.879ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
42862024-11-28T02:04:10.880ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
42872024-11-28T02:04:10.882ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
42882024-11-28T02:04:10.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
42892024-11-28T02:04:10.885ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
42902024-11-28T02:04:10.885ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
42912024-11-28T02:04:10.887ZINFOnexus (BackgroundTasks): VPC route manager sled 313847b3-d670-4167-9b09-9e7d3dfcb2a4 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
42922024-11-28T02:04:10.887ZDEBGnexus (BackgroundTasks): client request SledAgent = 313847b3-d670-4167-9b09-9e7d3dfcb2a4 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
42932024-11-28T02:04:10.888ZDEBGnexus (BackgroundTasks): client response SledAgent = 313847b3-d670-4167-9b09-9e7d3dfcb2a4 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "68cdd3f8-82e7-46c4-9761-587f3a198c4b", "date": "Thu, 28 Nov 2024 02:04:10 GMT"} })
42942024-11-28T02:04:10.890ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 410.601872ms iteration = 24 status = {}
42952024-11-28T02:04:10.986ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
42962024-11-28T02:04:11.026ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 547.095532ms iteration = 23 status = {}
42972024-11-28T02:04:11.034ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
42982024-11-28T02:04:11.034ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "4f3dc431-28d6-4475-b66c-2eb94585f232", "content-length": "1", "date": "Thu, 28 Nov 2024 02:04:11 GMT"} })
42992024-11-28T02:04:11.036ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
43002024-11-28T02:04:11.036ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
43012024-11-28T02:04:11.042ZDEBGnexus (BackgroundTasks): client request SledAgent = 313847b3-d670-4167-9b09-9e7d3dfcb2a4 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
43022024-11-28T02:04:11.044ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 564.999534ms iteration = 23 status = null
43032024-11-28T02:04:11.064ZDEBGnexus (BackgroundTasks): client response SledAgent = 313847b3-d670-4167-9b09-9e7d3dfcb2a4 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "9e823098-218d-4907-9b79-aef98df9cbc1", "content-length": "691", "date": "Thu, 28 Nov 2024 02:04:11 GMT"} })
43042024-11-28T02:04:11.064ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
43052024-11-28T02:04:11.064ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
43062024-11-28T02:04:11.064ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
43072024-11-28T02:04:11.066ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 586.539458ms iteration = 23 status = {}
43082024-11-28T02:04:11.089ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
43092024-11-28T02:04:11.089ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "11400ef4-e141-49be-83d2-3e84c7b4e0bc", "date": "Thu, 28 Nov 2024 02:04:10 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
43102024-11-28T02:04:11.089ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
43112024-11-28T02:04:11.092ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "b66cde20-0b6a-4075-a3ac-1054de8d10e3", "date": "Thu, 28 Nov 2024 02:04:10 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
43122024-11-28T02:04:11.092ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
43132024-11-28T02:04:11.093ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "bd546612-71e3-4555-9cb2-fcd93adf1323", "date": "Thu, 28 Nov 2024 02:04:10 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
43142024-11-28T02:04:11.093ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
43152024-11-28T02:04:11.094ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
43162024-11-28T02:04:11.094ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 615.322528ms iteration = 14 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
43172024-11-28T02:04:11.095ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
43182024-11-28T02:04:11.095ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
43192024-11-28T02:04:11.097ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 4bb2986f-2e92-4ac3-9dc1-591b6bb60b3e, rack_id: 92a8ff6e-21d3-4c28-bcad-e323847985dc, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(ea300318-9d62-42ed-80ea-21c86590a166) } rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc switch_location = Switch0
43202024-11-28T02:04:11.097ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
43212024-11-28T02:04:11.098ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
43222024-11-28T02:04:11.108ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
43232024-11-28T02:04:11.109ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "7dd41c2d-c55d-4a23-9052-550ef71598aa", "content-length": "134", "date": "Thu, 28 Nov 2024 02:04:11 GMT"} })
43242024-11-28T02:04:11.109ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1746 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
43252024-11-28T02:04:11.109ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1773 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = ea300318-9d62-42ed-80ea-21c86590a166
43262024-11-28T02:04:11.109ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
43272024-11-28T02:04:11.109ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
43282024-11-28T02:04:11.109ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
43292024-11-28T02:04:11.109ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
43302024-11-28T02:04:11.109ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
43312024-11-28T02:04:11.109ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1920 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
43322024-11-28T02:04:11.109ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
43332024-11-28T02:04:11.109ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
43342024-11-28T02:04:11.109ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc
43352024-11-28T02:04:11.109ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
43362024-11-28T02:04:11.110ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
43372024-11-28T02:04:11.112ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
43382024-11-28T02:04:11.112ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "da1f9a76-a6ce-4ae0-ab2f-91acede6e754", "content-length": "2", "date": "Thu, 28 Nov 2024 02:04:11 GMT"} })
43392024-11-28T02:04:11.112ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
43402024-11-28T02:04:11.112ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "091b1103-5d98-40e2-8643-e77b7b10e1fc", "content-length": "52", "date": "Thu, 28 Nov 2024 02:04:11 GMT"} })
43412024-11-28T02:04:11.112ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc switch_location = Switch0
43422024-11-28T02:04:11.112ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
43432024-11-28T02:04:11.229ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "58ae7ea8-0828-48b8-8fe4-b975ae18a299", "date": "Thu, 28 Nov 2024 02:04:11 GMT"} })
43442024-11-28T02:04:11.229ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc switch = Switch0
43452024-11-28T02:04:11.229ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
43462024-11-28T02:04:11.230ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "60c00010-2c4c-4844-965f-a9e17ff98bb7", "content-length": "682", "date": "Thu, 28 Nov 2024 02:04:11 GMT"} })
43472024-11-28T02:04:11.230ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
43482024-11-28T02:04:11.230ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
43492024-11-28T02:04:11.240ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
43502024-11-28T02:04:11.240ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
43512024-11-28T02:04:11.243ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
43522024-11-28T02:04:11.244ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
43532024-11-28T02:04:11.246ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
43542024-11-28T02:04:11.247ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
43552024-11-28T02:04:11.259ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 780.058809ms iteration = 23 status = {}
43562024-11-28T02:04:11.415ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:40402 req_id = aaa3bede-6da1-4475-ad08-7bc3d8ba85b0 roles = RoleSet { roles: {} } uri = /nat/ipv4/changeset/6?limit=100
43572024-11-28T02:04:11.416ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:40402 req_id = aaa3bede-6da1-4475-ad08-7bc3d8ba85b0 resource = Database result = Ok(()) uri = /nat/ipv4/changeset/6?limit=100
43582024-11-28T02:04:11.418ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2366 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:40402 req_id = aaa3bede-6da1-4475-ad08-7bc3d8ba85b0 response_code = 200 uri = /nat/ipv4/changeset/6?limit=100
43592024-11-28T02:04:11.851ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 2d4789b8-fd41-47aa-8d29-234b3bea3e87 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
43602024-11-28T02:04:11.852ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 2d4789b8-fd41-47aa-8d29-234b3bea3e87 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
43612024-11-28T02:04:11.853ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2008 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 2d4789b8-fd41-47aa-8d29-234b3bea3e87 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
43622024-11-28T02:04:12.874ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = ad33b51b-bf2f-451b-a4cd-cc77f16e3381 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
43632024-11-28T02:04:12.874ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = ad33b51b-bf2f-451b-a4cd-cc77f16e3381 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
43642024-11-28T02:04:12.876ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2738 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = ad33b51b-bf2f-451b-a4cd-cc77f16e3381 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
43652024-11-28T02:04:13.897ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 9bd8a1c5-2236-4628-9134-ad36e81fbe34 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
43662024-11-28T02:04:13.897ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 9bd8a1c5-2236-4628-9134-ad36e81fbe34 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
43672024-11-28T02:04:13.898ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1648 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 9bd8a1c5-2236-4628-9134-ad36e81fbe34 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
43682024-11-28T02:04:14.194ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1568 local_addr = [fd00:1122:3344:101::b]:55621 method = GET remote_addr = [fd00:1122:3344:101::d]:56417 req_id = d0e25473-606f-44af-a859-39cee257daaf response_code = 200 uri = /2469a748-bcd2-4e0d-99cc-f79def0452cc
43692024-11-28T02:04:14.919ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = f9ab5dc6-9627-4a6a-a48c-483134edb0d9 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
43702024-11-28T02:04:14.920ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = f9ab5dc6-9627-4a6a-a48c-483134edb0d9 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
43712024-11-28T02:04:14.923ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 3319 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = f9ab5dc6-9627-4a6a-a48c-483134edb0d9 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
43722024-11-28T02:04:15.943ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 5eb7f670-996a-433d-b697-7b0c9293dc0a roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
43732024-11-28T02:04:15.943ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 5eb7f670-996a-433d-b697-7b0c9293dc0a resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
43742024-11-28T02:04:15.945ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2245 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 5eb7f670-996a-433d-b697-7b0c9293dc0a response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
43752024-11-28T02:04:16.424ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:40402 req_id = 73503c16-91ea-46f1-992d-d3cfe87b7ed5 roles = RoleSet { roles: {} } uri = /nat/ipv4/changeset/6?limit=100
43762024-11-28T02:04:16.424ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:40402 req_id = 73503c16-91ea-46f1-992d-d3cfe87b7ed5 resource = Database result = Ok(()) uri = /nat/ipv4/changeset/6?limit=100
43772024-11-28T02:04:16.430ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 5948 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:40402 req_id = 73503c16-91ea-46f1-992d-d3cfe87b7ed5 response_code = 200 uri = /nat/ipv4/changeset/6?limit=100
43782024-11-28T02:04:16.431ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:40402 req_id = 2d6137d7-c0d9-4648-a787-5d318c91824a roles = RoleSet { roles: {} } uri = /nat/ipv4/changeset/8?limit=100
43792024-11-28T02:04:16.431ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:40402 req_id = 2d6137d7-c0d9-4648-a787-5d318c91824a resource = Database result = Ok(()) uri = /nat/ipv4/changeset/8?limit=100
43802024-11-28T02:04:16.434ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2871 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:40402 req_id = 2d6137d7-c0d9-4648-a787-5d318c91824a response_code = 200 uri = /nat/ipv4/changeset/8?limit=100
43812024-11-28T02:04:16.966ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = ed9bbd4b-189a-47a0-8525-305b5c46211a roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
43822024-11-28T02:04:16.966ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = ed9bbd4b-189a-47a0-8525-305b5c46211a resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
43832024-11-28T02:04:16.967ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1696 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = ed9bbd4b-189a-47a0-8525-305b5c46211a response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
43842024-11-28T02:04:17.986ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 85ceec02-1d6a-4250-b74f-7f7d1ed81cc9 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
43852024-11-28T02:04:17.987ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 85ceec02-1d6a-4250-b74f-7f7d1ed81cc9 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
43862024-11-28T02:04:17.989ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2483 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 85ceec02-1d6a-4250-b74f-7f7d1ed81cc9 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
43872024-11-28T02:04:19.010ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = bfc4e545-684a-470c-8ffe-1fc6a8ebd1b8 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
43882024-11-28T02:04:19.010ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = bfc4e545-684a-470c-8ffe-1fc6a8ebd1b8 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
43892024-11-28T02:04:19.013ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2527 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = bfc4e545-684a-470c-8ffe-1fc6a8ebd1b8 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
43902024-11-28T02:04:20.034ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = e85528dc-ba0e-48ff-b1c2-25727dbbf039 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
43912024-11-28T02:04:20.034ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = e85528dc-ba0e-48ff-b1c2-25727dbbf039 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
43922024-11-28T02:04:20.036ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2184 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = e85528dc-ba0e-48ff-b1c2-25727dbbf039 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
43932024-11-28T02:04:20.479ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 68 reason = Timeout
43942024-11-28T02:04:20.479ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 68 reason = Timeout
43952024-11-28T02:04:20.479ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
43962024-11-28T02:04:20.479ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
43972024-11-28T02:04:20.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
43982024-11-28T02:04:20.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
43992024-11-28T02:04:20.481ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
44002024-11-28T02:04:20.481ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
44012024-11-28T02:04:20.481ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
44022024-11-28T02:04:20.484ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
44032024-11-28T02:04:20.484ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
44042024-11-28T02:04:20.484ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
44052024-11-28T02:04:20.486ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
44062024-11-28T02:04:20.486ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
44072024-11-28T02:04:20.488ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
44082024-11-28T02:04:20.491ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 67ac247f-c934-4c78-857a-fba4fbeb9937, lookup_type: ById(67ac247f-c934-4c78-857a-fba4fbeb9937) } result = Ok(())
44092024-11-28T02:04:20.491ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
44102024-11-28T02:04:20.491ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
44112024-11-28T02:04:20.502ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 67ac247f-c934-4c78-857a-fba4fbeb9937 original_time_created = 2024-11-28 01:53:03.159475 UTC target_id = 67ac247f-c934-4c78-857a-fba4fbeb9937 time_created = 2024-11-28 01:53:03.159475 UTC
44122024-11-28T02:04:20.502ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 23.519698ms iteration = 68 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"67ac247f-c934-4c78-857a-fba4fbeb9937","time_created":"2024-11-28T01:53:03.159475Z"}
44132024-11-28T02:04:20.511ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 31.909264ms iteration = 68 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
44142024-11-28T02:04:21.057ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 1d513cee-f467-405f-9256-3fc9c4ba477b roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
44152024-11-28T02:04:21.057ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 1d513cee-f467-405f-9256-3fc9c4ba477b resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
44162024-11-28T02:04:21.058ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1623 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 1d513cee-f467-405f-9256-3fc9c4ba477b response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
44172024-11-28T02:04:22.080ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = c7048b59-2767-4f0f-9f85-da4540dfedd1 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
44182024-11-28T02:04:22.081ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = c7048b59-2767-4f0f-9f85-da4540dfedd1 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
44192024-11-28T02:04:22.083ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2646 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = c7048b59-2767-4f0f-9f85-da4540dfedd1 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
44202024-11-28T02:04:23.102ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 6183097c-0a56-4d6a-89d4-26ddc9007308 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
44212024-11-28T02:04:23.103ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 6183097c-0a56-4d6a-89d4-26ddc9007308 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
44222024-11-28T02:04:23.104ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2001 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 6183097c-0a56-4d6a-89d4-26ddc9007308 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
44232024-11-28T02:04:24.126ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = fab00eb8-59cc-442a-8c0f-57788d3d7955 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
44242024-11-28T02:04:24.127ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = fab00eb8-59cc-442a-8c0f-57788d3d7955 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
44252024-11-28T02:04:24.129ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2631 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = fab00eb8-59cc-442a-8c0f-57788d3d7955 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
44262024-11-28T02:04:24.194ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1187 local_addr = [fd00:1122:3344:101::b]:55621 method = GET remote_addr = [fd00:1122:3344:101::d]:56417 req_id = f449046e-4641-4856-ac5c-46254e555611 response_code = 200 uri = /2469a748-bcd2-4e0d-99cc-f79def0452cc
44272024-11-28T02:04:24.263ZINFOnexus (dropshot_internal): received new VMM runtime state from sled agent actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true file = nexus/src/app/instance.rs:2116 local_addr = [fd00:1122:3344:101::b]:12221 method = PUT migration_state = Migrations { migration_in: None, migration_out: None } propolis_id = 7001f3a3-3a22-4fd6-a06f-efde5d6ec855 remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 77e4c3c0-e2ba-4afc-a7b3-9fe879d1bfab uri = /vmms/7001f3a3-3a22-4fd6-a06f-efde5d6ec855 vmm_state = VmmRuntimeState { state: Starting, gen: Generation(3), time_updated: 2024-11-28T02:04:24.260553404Z }
44282024-11-28T02:04:24.263ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 77e4c3c0-e2ba-4afc-a7b3-9fe879d1bfab roles = RoleSet { roles: {} } uri = /vmms/7001f3a3-3a22-4fd6-a06f-efde5d6ec855
44292024-11-28T02:04:24.263ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 77e4c3c0-e2ba-4afc-a7b3-9fe879d1bfab resource = Database result = Ok(()) uri = /vmms/7001f3a3-3a22-4fd6-a06f-efde5d6ec855
44302024-11-28T02:04:24.293ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::b]:12221 remote_addr = [fd00:1122:3344:101::1:0]:49250
44312024-11-28T02:04:24.294ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::1:0]:49250 req_id = 3bc30c3e-afb1-424b-858b-37fb4859fb27 roles = RoleSet { roles: {} } uri = /metrics/producers
44322024-11-28T02:04:24.294ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::1:0]:49250 req_id = 3bc30c3e-afb1-424b-858b-37fb4859fb27 resource = Database result = Ok(()) uri = /metrics/producers
44332024-11-28T02:04:24.304ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 43440 local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 77e4c3c0-e2ba-4afc-a7b3-9fe879d1bfab response_code = 204 uri = /vmms/7001f3a3-3a22-4fd6-a06f-efde5d6ec855
44342024-11-28T02:04:24.305ZINFOnexus (dropshot_internal): received new VMM runtime state from sled agent actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true file = nexus/src/app/instance.rs:2116 local_addr = [fd00:1122:3344:101::b]:12221 method = PUT migration_state = Migrations { migration_in: None, migration_out: None } propolis_id = 7001f3a3-3a22-4fd6-a06f-efde5d6ec855 remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 2189b604-c57b-41b2-bf83-f2433b156d8c uri = /vmms/7001f3a3-3a22-4fd6-a06f-efde5d6ec855 vmm_state = VmmRuntimeState { state: Running, gen: Generation(4), time_updated: 2024-11-28T02:04:24.305107638Z }
44352024-11-28T02:04:24.305ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 2189b604-c57b-41b2-bf83-f2433b156d8c roles = RoleSet { roles: {} } uri = /vmms/7001f3a3-3a22-4fd6-a06f-efde5d6ec855
44362024-11-28T02:04:24.305ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 2189b604-c57b-41b2-bf83-f2433b156d8c resource = Database result = Ok(()) uri = /vmms/7001f3a3-3a22-4fd6-a06f-efde5d6ec855
44372024-11-28T02:04:24.312ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 7676 local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 2189b604-c57b-41b2-bf83-f2433b156d8c response_code = 204 uri = /vmms/7001f3a3-3a22-4fd6-a06f-efde5d6ec855
44382024-11-28T02:04:24.329ZINFOnexus: assigned collector to new producer collector_id = 119eed64-2e24-4d5f-8c42-be6cd8bec8b5 file = nexus/src/app/oximeter.rs:76 producer_id = 754b28bf-6c2b-4d04-be10-941b3bfa03d8
44392024-11-28T02:04:24.329ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 35728 local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::1:0]:49250 req_id = 3bc30c3e-afb1-424b-858b-37fb4859fb27 response_code = 201 uri = /metrics/producers
44402024-11-28T02:04:25.148ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = bf82f791-d45e-486e-acd8-b72eef0ecf34 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
44412024-11-28T02:04:25.148ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = bf82f791-d45e-486e-acd8-b72eef0ecf34 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
44422024-11-28T02:04:25.150ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2436 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = bf82f791-d45e-486e-acd8-b72eef0ecf34 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
44432024-11-28T02:04:26.169ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 1b832bba-0575-4005-83fa-321fe21a6304 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
44442024-11-28T02:04:26.170ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 1b832bba-0575-4005-83fa-321fe21a6304 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
44452024-11-28T02:04:26.172ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2399 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 1b832bba-0575-4005-83fa-321fe21a6304 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
44462024-11-28T02:04:27.191ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 407f74d4-ae90-430b-9881-98abaf6fa035 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
44472024-11-28T02:04:27.192ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 407f74d4-ae90-430b-9881-98abaf6fa035 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
44482024-11-28T02:04:27.194ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2563 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 407f74d4-ae90-430b-9881-98abaf6fa035 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
44492024-11-28T02:04:28.215ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = c0078b0c-b285-4f49-aeed-424c6dfab83d roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
44502024-11-28T02:04:28.215ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = c0078b0c-b285-4f49-aeed-424c6dfab83d resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
44512024-11-28T02:04:28.217ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1927 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = c0078b0c-b285-4f49-aeed-424c6dfab83d response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
44522024-11-28T02:04:29.236ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 160e8d9c-4ab9-4a8b-81f0-3e1fc6c21b39 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
44532024-11-28T02:04:29.237ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 160e8d9c-4ab9-4a8b-81f0-3e1fc6c21b39 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
44542024-11-28T02:04:29.239ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2495 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 160e8d9c-4ab9-4a8b-81f0-3e1fc6c21b39 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
44552024-11-28T02:04:30.257ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 2d62137c-6e06-4963-a694-2ec6beb0c834 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
44562024-11-28T02:04:30.257ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 2d62137c-6e06-4963-a694-2ec6beb0c834 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
44572024-11-28T02:04:30.259ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2194 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 2d62137c-6e06-4963-a694-2ec6beb0c834 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
44582024-11-28T02:04:30.478ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 69 reason = Timeout
44592024-11-28T02:04:30.478ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 69 reason = Timeout
44602024-11-28T02:04:30.478ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
44612024-11-28T02:04:30.478ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
44622024-11-28T02:04:30.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
44632024-11-28T02:04:30.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
44642024-11-28T02:04:30.481ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
44652024-11-28T02:04:30.481ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
44662024-11-28T02:04:30.481ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
44672024-11-28T02:04:30.483ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
44682024-11-28T02:04:30.483ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
44692024-11-28T02:04:30.484ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
44702024-11-28T02:04:30.486ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
44712024-11-28T02:04:30.486ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
44722024-11-28T02:04:30.488ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
44732024-11-28T02:04:30.490ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 67ac247f-c934-4c78-857a-fba4fbeb9937, lookup_type: ById(67ac247f-c934-4c78-857a-fba4fbeb9937) } result = Ok(())
44742024-11-28T02:04:30.490ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
44752024-11-28T02:04:30.490ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
44762024-11-28T02:04:30.499ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 67ac247f-c934-4c78-857a-fba4fbeb9937 original_time_created = 2024-11-28 01:53:03.159475 UTC target_id = 67ac247f-c934-4c78-857a-fba4fbeb9937 time_created = 2024-11-28 01:53:03.159475 UTC
44772024-11-28T02:04:30.499ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 21.181028ms iteration = 69 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"67ac247f-c934-4c78-857a-fba4fbeb9937","time_created":"2024-11-28T01:53:03.159475Z"}
44782024-11-28T02:04:30.510ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 31.709751ms iteration = 69 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
44792024-11-28T02:04:31.279ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 20f9ca13-9b1a-4d6e-ac12-06f2235e5cfb roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
44802024-11-28T02:04:31.279ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 20f9ca13-9b1a-4d6e-ac12-06f2235e5cfb resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
44812024-11-28T02:04:31.281ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2212 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 20f9ca13-9b1a-4d6e-ac12-06f2235e5cfb response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
44822024-11-28T02:04:32.301ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 250097d3-e646-4d26-b647-83e8c8e8e796 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
44832024-11-28T02:04:32.302ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 250097d3-e646-4d26-b647-83e8c8e8e796 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
44842024-11-28T02:04:32.303ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2265 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 250097d3-e646-4d26-b647-83e8c8e8e796 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
44852024-11-28T02:04:33.322ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 89f56bc5-6c3e-4adb-832f-b450d7f55832 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
44862024-11-28T02:04:33.322ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 89f56bc5-6c3e-4adb-832f-b450d7f55832 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
44872024-11-28T02:04:33.324ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2187 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 89f56bc5-6c3e-4adb-832f-b450d7f55832 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
44882024-11-28T02:04:34.195ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1859 local_addr = [fd00:1122:3344:101::b]:55621 method = GET remote_addr = [fd00:1122:3344:101::d]:56417 req_id = 96371226-186a-4614-bf52-34c946e28d20 response_code = 200 uri = /2469a748-bcd2-4e0d-99cc-f79def0452cc
44892024-11-28T02:04:34.343ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 55e75868-a3e9-4601-bf57-1a2133587f09 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
44902024-11-28T02:04:34.343ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 55e75868-a3e9-4601-bf57-1a2133587f09 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
44912024-11-28T02:04:34.345ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1828 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 55e75868-a3e9-4601-bf57-1a2133587f09 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
44922024-11-28T02:04:34.429ZINFOnexus (dropshot_internal): received new VMM runtime state from sled agent actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true file = nexus/src/app/instance.rs:2116 local_addr = [fd00:1122:3344:101::b]:12221 method = PUT migration_state = Migrations { migration_in: None, migration_out: None } propolis_id = 7001f3a3-3a22-4fd6-a06f-efde5d6ec855 remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 6151dfa7-8fc9-4b72-91e6-bd47082a7443 uri = /vmms/7001f3a3-3a22-4fd6-a06f-efde5d6ec855 vmm_state = VmmRuntimeState { state: Stopping, gen: Generation(6), time_updated: 2024-11-28T02:04:34.428798846Z }
44932024-11-28T02:04:34.429ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 6151dfa7-8fc9-4b72-91e6-bd47082a7443 roles = RoleSet { roles: {} } uri = /vmms/7001f3a3-3a22-4fd6-a06f-efde5d6ec855
44942024-11-28T02:04:34.429ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 6151dfa7-8fc9-4b72-91e6-bd47082a7443 resource = Database result = Ok(()) uri = /vmms/7001f3a3-3a22-4fd6-a06f-efde5d6ec855
44952024-11-28T02:04:34.450ZWARNnexus (datastore): Retryable transaction failure file = nexus/db-queries/src/transaction_retry.rs:159 retry_after (ms) = 25 transaction = vmm_and_migration_update_runtime
44962024-11-28T02:04:34.489ZINFOnexus (datastore): transaction completed attempts = 2 file = nexus/db-queries/src/transaction_retry.rs:115 transaction = vmm_and_migration_update_runtime
44972024-11-28T02:04:34.489ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 60402 local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 6151dfa7-8fc9-4b72-91e6-bd47082a7443 response_code = 204 uri = /vmms/7001f3a3-3a22-4fd6-a06f-efde5d6ec855
44982024-11-28T02:04:35.364ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = c54b0812-c566-44b5-b45f-be495b4f533c roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
44992024-11-28T02:04:35.365ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = c54b0812-c566-44b5-b45f-be495b4f533c resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
45002024-11-28T02:04:35.367ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2800 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = c54b0812-c566-44b5-b45f-be495b4f533c response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
45012024-11-28T02:04:35.453ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:40402 req_id = bc9e7eea-8c34-4682-a900-fa5c2ab97084 roles = RoleSet { roles: {} } uri = /nat/ipv4/changeset/8?limit=100
45022024-11-28T02:04:35.453ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:40402 req_id = bc9e7eea-8c34-4682-a900-fa5c2ab97084 resource = Database result = Ok(()) uri = /nat/ipv4/changeset/8?limit=100
45032024-11-28T02:04:35.457ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 3736 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:40402 req_id = bc9e7eea-8c34-4682-a900-fa5c2ab97084 response_code = 200 uri = /nat/ipv4/changeset/8?limit=100
45042024-11-28T02:04:35.457ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:40402 req_id = e26f7c32-bd1f-4a1d-9479-0551554b1375 roles = RoleSet { roles: {} } uri = /nat/ipv4/changeset/10?limit=100
45052024-11-28T02:04:35.458ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:40402 req_id = e26f7c32-bd1f-4a1d-9479-0551554b1375 resource = Database result = Ok(()) uri = /nat/ipv4/changeset/10?limit=100
45062024-11-28T02:04:35.461ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 3472 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:40402 req_id = e26f7c32-bd1f-4a1d-9479-0551554b1375 response_code = 200 uri = /nat/ipv4/changeset/10?limit=100
45072024-11-28T02:04:36.386ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 78f71e1a-68a6-4004-910a-d5ce244eb7c2 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
45082024-11-28T02:04:36.386ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 78f71e1a-68a6-4004-910a-d5ce244eb7c2 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
45092024-11-28T02:04:36.388ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2352 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 78f71e1a-68a6-4004-910a-d5ce244eb7c2 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
45102024-11-28T02:04:36.462ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:40402 req_id = 04a63e25-003a-4858-9f6e-b9eb7c9ec6e8 roles = RoleSet { roles: {} } uri = /nat/ipv4/changeset/10?limit=100
45112024-11-28T02:04:36.463ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:40402 req_id = 04a63e25-003a-4858-9f6e-b9eb7c9ec6e8 resource = Database result = Ok(()) uri = /nat/ipv4/changeset/10?limit=100
45122024-11-28T02:04:36.466ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 3257 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:40402 req_id = 04a63e25-003a-4858-9f6e-b9eb7c9ec6e8 response_code = 200 uri = /nat/ipv4/changeset/10?limit=100
45132024-11-28T02:04:36.911ZINFOnexus (dropshot_internal): received new VMM runtime state from sled agent actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true file = nexus/src/app/instance.rs:2116 local_addr = [fd00:1122:3344:101::b]:12221 method = PUT migration_state = Migrations { migration_in: None, migration_out: None } propolis_id = 7001f3a3-3a22-4fd6-a06f-efde5d6ec855 remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 05996897-6f33-4bc0-bd3c-38b993c367a6 uri = /vmms/7001f3a3-3a22-4fd6-a06f-efde5d6ec855 vmm_state = VmmRuntimeState { state: Destroyed, gen: Generation(8), time_updated: 2024-11-28T02:04:36.911491548Z }
45142024-11-28T02:04:36.911ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 05996897-6f33-4bc0-bd3c-38b993c367a6 roles = RoleSet { roles: {} } uri = /vmms/7001f3a3-3a22-4fd6-a06f-efde5d6ec855
45152024-11-28T02:04:36.912ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 05996897-6f33-4bc0-bd3c-38b993c367a6 resource = Database result = Ok(()) uri = /vmms/7001f3a3-3a22-4fd6-a06f-efde5d6ec855
45162024-11-28T02:04:36.923ZDEBGnexus (dropshot_internal): new VMM runtime state from sled agent requires an instance-update saga actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT migration_in_needs_update = false migration_out_needs_update = false propolis_id = 7001f3a3-3a22-4fd6-a06f-efde5d6ec855 remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 05996897-6f33-4bc0-bd3c-38b993c367a6 uri = /vmms/7001f3a3-3a22-4fd6-a06f-efde5d6ec855 vmm_needs_update = true
45172024-11-28T02:04:36.923ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 05996897-6f33-4bc0-bd3c-38b993c367a6 roles = RoleSet { roles: {} } uri = /vmms/7001f3a3-3a22-4fd6-a06f-efde5d6ec855
45182024-11-28T02:04:36.923ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 05996897-6f33-4bc0-bd3c-38b993c367a6 resource = Database result = Ok(()) uri = /vmms/7001f3a3-3a22-4fd6-a06f-efde5d6ec855
45192024-11-28T02:04:36.925ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 05996897-6f33-4bc0-bd3c-38b993c367a6 roles = RoleSet { roles: {} } uri = /vmms/7001f3a3-3a22-4fd6-a06f-efde5d6ec855
45202024-11-28T02:04:36.926ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 05996897-6f33-4bc0-bd3c-38b993c367a6 resource = Database result = Ok(()) uri = /vmms/7001f3a3-3a22-4fd6-a06f-efde5d6ec855
45212024-11-28T02:04:36.927ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 05996897-6f33-4bc0-bd3c-38b993c367a6 roles = RoleSet { roles: {} } uri = /vmms/7001f3a3-3a22-4fd6-a06f-efde5d6ec855
45222024-11-28T02:04:36.927ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 05996897-6f33-4bc0-bd3c-38b993c367a6 resource = Database result = Ok(()) uri = /vmms/7001f3a3-3a22-4fd6-a06f-efde5d6ec855
45232024-11-28T02:04:36.930ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 05996897-6f33-4bc0-bd3c-38b993c367a6 roles = RoleSet { roles: {} } uri = /vmms/7001f3a3-3a22-4fd6-a06f-efde5d6ec855
45242024-11-28T02:04:36.930ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 05996897-6f33-4bc0-bd3c-38b993c367a6 resource = Database result = Ok(()) uri = /vmms/7001f3a3-3a22-4fd6-a06f-efde5d6ec855
45252024-11-28T02:04:36.931ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 05996897-6f33-4bc0-bd3c-38b993c367a6 roles = RoleSet { roles: {} } uri = /vmms/7001f3a3-3a22-4fd6-a06f-efde5d6ec855
45262024-11-28T02:04:36.932ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 05996897-6f33-4bc0-bd3c-38b993c367a6 resource = Database result = Ok(()) uri = /vmms/7001f3a3-3a22-4fd6-a06f-efde5d6ec855
45272024-11-28T02:04:36.933ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 05996897-6f33-4bc0-bd3c-38b993c367a6 roles = RoleSet { roles: {} } uri = /vmms/7001f3a3-3a22-4fd6-a06f-efde5d6ec855
45282024-11-28T02:04:36.933ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 05996897-6f33-4bc0-bd3c-38b993c367a6 resource = Database result = Ok(()) uri = /vmms/7001f3a3-3a22-4fd6-a06f-efde5d6ec855
45292024-11-28T02:04:36.934ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 05996897-6f33-4bc0-bd3c-38b993c367a6 roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } uri = /vmms/7001f3a3-3a22-4fd6-a06f-efde5d6ec855
45302024-11-28T02:04:36.937ZDEBGnexus (dropshot_internal): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 05996897-6f33-4bc0-bd3c-38b993c367a6 resource = Instance { parent: Project { parent: Silo { parent: Fleet, key: 7e159834-997a-4c03-b9db-1e91d783221c, lookup_type: ById(7e159834-997a-4c03-b9db-1e91d783221c) }, key: 4e06baac-07fc-4792-8ebe-158a470fd7b6, lookup_type: ById(4e06baac-07fc-4792-8ebe-158a470fd7b6) }, key: 754b28bf-6c2b-4d04-be10-941b3bfa03d8, lookup_type: ById(754b28bf-6c2b-4d04-be10-941b3bfa03d8) } result = Ok(()) uri = /vmms/7001f3a3-3a22-4fd6-a06f-efde5d6ec855
45312024-11-28T02:04:36.937ZINFOnexus (dropshot_internal): starting update saga for 754b28bf-6c2b-4d04-be10-941b3bfa03d8 actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true file = nexus/src/app/instance.rs:1583 instance_id = 754b28bf-6c2b-4d04-be10-941b3bfa03d8 local_addr = [fd00:1122:3344:101::b]:12221 method = PUT migration_state = Migrations { migration_in: None, migration_out: None } remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 05996897-6f33-4bc0-bd3c-38b993c367a6 uri = /vmms/7001f3a3-3a22-4fd6-a06f-efde5d6ec855 vmm_state = VmmRuntimeState { state: Destroyed, gen: Generation(8), time_updated: 2024-11-28T02:04:36.911491548Z }
45322024-11-28T02:04:36.937ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 25675 local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 05996897-6f33-4bc0-bd3c-38b993c367a6 response_code = 204 uri = /vmms/7001f3a3-3a22-4fd6-a06f-efde5d6ec855
45332024-11-28T02:04:36.937ZDEBGnexus (dropshot_internal): preparing instance-update saga for 754b28bf-6c2b-4d04-be10-941b3bfa03d8... actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true instance_id = 754b28bf-6c2b-4d04-be10-941b3bfa03d8 local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 05996897-6f33-4bc0-bd3c-38b993c367a6 uri = /vmms/7001f3a3-3a22-4fd6-a06f-efde5d6ec855
45342024-11-28T02:04:36.937ZINFOnexus (SagaExecutor): preparing saga file = nexus/src/app/saga.rs:255 saga_id = 8f55e372-61a7-41a4-a363-d8a319d45da5 saga_name = start-instance-update
45352024-11-28T02:04:36.937ZINFOnexus (SEC): saga create dag = {"end_node":4,"graph":{"edge_property":"directed","edges":[[0,1,null],[1,2,null],[3,0,null],[2,4,null]],"node_holes":[],"nodes":[{"Action":{"action_name":"common.uuid_generate","label":"GenerateInstanceLockId","name":"saga_instance_lock_id"}},{"Action":{"action_name":"start_instance_update.lock_instance","label":"LockInstance","name":"updater_lock"}},{"Action":{"action_name":"start_instance_update.fetch_state_and_start_real_saga","label":"FetchStateAndStartRealSaga","name":"state"}},{"Start":{"params":{"authz_instance":{"key":"754b28bf-6c2b-4d04-be10-941b3bfa03d8","lookup_type":{"ById":"754b28bf-6c2b-4d04-be10-941b3bfa03d8"},"parent":{"key":"4e06baac-07fc-4792-8ebe-158a470fd7b6","lookup_type":{"ById":"4e06baac-07fc-4792-8ebe-158a470fd7b6"},"parent":{"key":"7e159834-997a-4c03-b9db-1e91d783221c","lookup_type":{"ById":"7e159834-997a-4c03-b9db-1e91d783221c"},"parent":null}}},"serialized_authn":{"kind":{"Authenticated":[{"actor":{"UserBuiltin":{"user_builtin_id":"001de000-05e4-4000-8000-000000000002"}}},null]}}}}},"End"]},"saga_name":"start-instance-update","start_node":3} file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/steno-0.4.1/src/sec.rs:1146 saga_id = 8f55e372-61a7-41a4-a363-d8a319d45da5 saga_name = start-instance-update sec_id = 2469a748-bcd2-4e0d-99cc-f79def0452cc
45362024-11-28T02:04:36.937ZINFOnexus (SecStore): creating saga file = nexus/db-queries/src/db/sec_store.rs:50 saga_id = 8f55e372-61a7-41a4-a363-d8a319d45da5 saga_name = start-instance-update
45372024-11-28T02:04:36.937ZINFOnexus (dropshot_internal): received new VMM runtime state from sled agent actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true file = nexus/src/app/instance.rs:2116 local_addr = [fd00:1122:3344:101::b]:12221 method = PUT migration_state = Migrations { migration_in: None, migration_out: None } propolis_id = 7001f3a3-3a22-4fd6-a06f-efde5d6ec855 remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 3525278a-264e-4457-9b73-cb6cbd454d10 uri = /vmms/7001f3a3-3a22-4fd6-a06f-efde5d6ec855 vmm_state = VmmRuntimeState { state: Destroyed, gen: Generation(8), time_updated: 2024-11-28T02:04:36.911491548Z }
45382024-11-28T02:04:36.937ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 3525278a-264e-4457-9b73-cb6cbd454d10 roles = RoleSet { roles: {} } uri = /vmms/7001f3a3-3a22-4fd6-a06f-efde5d6ec855
45392024-11-28T02:04:36.938ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 3525278a-264e-4457-9b73-cb6cbd454d10 resource = Database result = Ok(()) uri = /vmms/7001f3a3-3a22-4fd6-a06f-efde5d6ec855
45402024-11-28T02:04:36.969ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 31704 local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 3525278a-264e-4457-9b73-cb6cbd454d10 response_code = 204 uri = /vmms/7001f3a3-3a22-4fd6-a06f-efde5d6ec855
45412024-11-28T02:04:36.975ZINFOnexus (SagaExecutor): starting saga file = nexus/src/app/saga.rs:353 saga_id = 8f55e372-61a7-41a4-a363-d8a319d45da5 saga_name = start-instance-update
45422024-11-28T02:04:36.975ZINFOnexus (SEC): saga start file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/steno-0.4.1/src/sec.rs:967 saga_id = 8f55e372-61a7-41a4-a363-d8a319d45da5 saga_name = start-instance-update sec_id = 2469a748-bcd2-4e0d-99cc-f79def0452cc
45432024-11-28T02:04:36.975ZDEBGnexus (SEC): saga log event new_state = N003 started sec_id = 2469a748-bcd2-4e0d-99cc-f79def0452cc
45442024-11-28T02:04:36.975ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 3 saga_id = 8f55e372-61a7-41a4-a363-d8a319d45da5
45452024-11-28T02:04:36.988ZDEBGnexus (SEC): saga log event new_state = N003 succeeded sec_id = 2469a748-bcd2-4e0d-99cc-f79def0452cc
45462024-11-28T02:04:36.988ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 3 saga_id = 8f55e372-61a7-41a4-a363-d8a319d45da5
45472024-11-28T02:04:37.001ZDEBGnexus (SEC): saga log event new_state = N000 started sec_id = 2469a748-bcd2-4e0d-99cc-f79def0452cc
45482024-11-28T02:04:37.001ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 0 saga_id = 8f55e372-61a7-41a4-a363-d8a319d45da5
45492024-11-28T02:04:37.006ZDEBGnexus (SEC): saga log event new_state = N000 succeeded sec_id = 2469a748-bcd2-4e0d-99cc-f79def0452cc
45502024-11-28T02:04:37.006ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(String("3e9e8414-5ef7-4e8a-8104-2e7206cb01a3")) node_id = 0 saga_id = 8f55e372-61a7-41a4-a363-d8a319d45da5
45512024-11-28T02:04:37.011ZDEBGnexus (SEC): saga log event new_state = N001 started sec_id = 2469a748-bcd2-4e0d-99cc-f79def0452cc
45522024-11-28T02:04:37.011ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 1 saga_id = 8f55e372-61a7-41a4-a363-d8a319d45da5
45532024-11-28T02:04:37.018ZINFOnexus (SagaExecutor): instance update: attempting to lock instance file = nexus/src/app/sagas/instance_update/start.rs:93 instance_id = 754b28bf-6c2b-4d04-be10-941b3bfa03d8 saga_id = 8f55e372-61a7-41a4-a363-d8a319d45da5 saga_name = start-instance-update
45542024-11-28T02:04:37.018ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 8f55e372-61a7-41a4-a363-d8a319d45da5 saga_name = start-instance-update saga_node = LockInstance
45552024-11-28T02:04:37.018ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 8f55e372-61a7-41a4-a363-d8a319d45da5 saga_name = start-instance-update saga_node = LockInstance
45562024-11-28T02:04:37.020ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 8f55e372-61a7-41a4-a363-d8a319d45da5 saga_name = start-instance-update saga_node = LockInstance
45572024-11-28T02:04:37.021ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 8f55e372-61a7-41a4-a363-d8a319d45da5 saga_name = start-instance-update saga_node = LockInstance
45582024-11-28T02:04:37.022ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 8f55e372-61a7-41a4-a363-d8a319d45da5 saga_name = start-instance-update saga_node = LockInstance
45592024-11-28T02:04:37.022ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 8f55e372-61a7-41a4-a363-d8a319d45da5 saga_name = start-instance-update saga_node = LockInstance
45602024-11-28T02:04:37.024ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 8f55e372-61a7-41a4-a363-d8a319d45da5 saga_name = start-instance-update saga_node = LockInstance
45612024-11-28T02:04:37.024ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 8f55e372-61a7-41a4-a363-d8a319d45da5 saga_name = start-instance-update saga_node = LockInstance
45622024-11-28T02:04:37.026ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 8f55e372-61a7-41a4-a363-d8a319d45da5 saga_name = start-instance-update saga_node = LockInstance
45632024-11-28T02:04:37.026ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 8f55e372-61a7-41a4-a363-d8a319d45da5 saga_name = start-instance-update saga_node = LockInstance
45642024-11-28T02:04:37.028ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 8f55e372-61a7-41a4-a363-d8a319d45da5 saga_name = start-instance-update saga_node = LockInstance
45652024-11-28T02:04:37.028ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 8f55e372-61a7-41a4-a363-d8a319d45da5 saga_name = start-instance-update saga_node = LockInstance
45662024-11-28T02:04:37.029ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } saga_id = 8f55e372-61a7-41a4-a363-d8a319d45da5 saga_name = start-instance-update saga_node = LockInstance
45672024-11-28T02:04:37.032ZDEBGnexus (SagaExecutor): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Instance { parent: Project { parent: Silo { parent: Fleet, key: 7e159834-997a-4c03-b9db-1e91d783221c, lookup_type: ById(7e159834-997a-4c03-b9db-1e91d783221c) }, key: 4e06baac-07fc-4792-8ebe-158a470fd7b6, lookup_type: ById(4e06baac-07fc-4792-8ebe-158a470fd7b6) }, key: 754b28bf-6c2b-4d04-be10-941b3bfa03d8, lookup_type: ById(754b28bf-6c2b-4d04-be10-941b3bfa03d8) } result = Ok(()) saga_id = 8f55e372-61a7-41a4-a363-d8a319d45da5 saga_name = start-instance-update saga_node = LockInstance
45682024-11-28T02:04:37.032ZDEBGnexus (SagaExecutor): attempting to acquire instance updater lock actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true current_gen = Generation(Generation(1)) instance_id = 754b28bf-6c2b-4d04-be10-941b3bfa03d8 saga_id = 8f55e372-61a7-41a4-a363-d8a319d45da5 saga_name = start-instance-update saga_node = LockInstance updater_id = 3e9e8414-5ef7-4e8a-8104-2e7206cb01a3
45692024-11-28T02:04:37.032ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 8f55e372-61a7-41a4-a363-d8a319d45da5 saga_name = start-instance-update saga_node = LockInstance
45702024-11-28T02:04:37.032ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 8f55e372-61a7-41a4-a363-d8a319d45da5 saga_name = start-instance-update saga_node = LockInstance
45712024-11-28T02:04:37.048ZDEBGnexus (SagaExecutor): attempting to acquire instance updater lock actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true current_gen = Generation(Generation(1)) instance_id = 754b28bf-6c2b-4d04-be10-941b3bfa03d8 saga_id = 8f55e372-61a7-41a4-a363-d8a319d45da5 saga_name = start-instance-update saga_node = LockInstance updater_id = 3e9e8414-5ef7-4e8a-8104-2e7206cb01a3
45722024-11-28T02:04:37.048ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 8f55e372-61a7-41a4-a363-d8a319d45da5 saga_name = start-instance-update saga_node = LockInstance
45732024-11-28T02:04:37.048ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 8f55e372-61a7-41a4-a363-d8a319d45da5 saga_name = start-instance-update saga_node = LockInstance
45742024-11-28T02:04:37.055ZDEBGnexus (SagaExecutor): instance updater lock acquired! actor_id = 001de000-05e4-4000-8000-000000000002 already_locked = true authenticated = true instance_id = 754b28bf-6c2b-4d04-be10-941b3bfa03d8 locked_gen = Generation(Generation(2)) saga_id = 8f55e372-61a7-41a4-a363-d8a319d45da5 saga_name = start-instance-update saga_node = LockInstance updater_id = 3e9e8414-5ef7-4e8a-8104-2e7206cb01a3
45752024-11-28T02:04:37.055ZDEBGnexus (SEC): saga log event new_state = N001 succeeded sec_id = 2469a748-bcd2-4e0d-99cc-f79def0452cc
45762024-11-28T02:04:37.055ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Object {"locked_gen": Number(2), "updater_id": String("3e9e8414-5ef7-4e8a-8104-2e7206cb01a3")}) node_id = 1 saga_id = 8f55e372-61a7-41a4-a363-d8a319d45da5
45772024-11-28T02:04:37.060ZDEBGnexus (SEC): saga log event new_state = N002 started sec_id = 2469a748-bcd2-4e0d-99cc-f79def0452cc
45782024-11-28T02:04:37.060ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 2 saga_id = 8f55e372-61a7-41a4-a363-d8a319d45da5
45792024-11-28T02:04:37.065ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 8f55e372-61a7-41a4-a363-d8a319d45da5 saga_name = start-instance-update saga_node = FetchStateAndStartRealSaga
45802024-11-28T02:04:37.065ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 8f55e372-61a7-41a4-a363-d8a319d45da5 saga_name = start-instance-update saga_node = FetchStateAndStartRealSaga
45812024-11-28T02:04:37.066ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 8f55e372-61a7-41a4-a363-d8a319d45da5 saga_name = start-instance-update saga_node = FetchStateAndStartRealSaga
45822024-11-28T02:04:37.066ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 8f55e372-61a7-41a4-a363-d8a319d45da5 saga_name = start-instance-update saga_node = FetchStateAndStartRealSaga
45832024-11-28T02:04:37.068ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 8f55e372-61a7-41a4-a363-d8a319d45da5 saga_name = start-instance-update saga_node = FetchStateAndStartRealSaga
45842024-11-28T02:04:37.068ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 8f55e372-61a7-41a4-a363-d8a319d45da5 saga_name = start-instance-update saga_node = FetchStateAndStartRealSaga
45852024-11-28T02:04:37.069ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } saga_id = 8f55e372-61a7-41a4-a363-d8a319d45da5 saga_name = start-instance-update saga_node = FetchStateAndStartRealSaga
45862024-11-28T02:04:37.072ZDEBGnexus (SagaExecutor): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Instance { parent: Project { parent: Silo { parent: Fleet, key: 7e159834-997a-4c03-b9db-1e91d783221c, lookup_type: ById(7e159834-997a-4c03-b9db-1e91d783221c) }, key: 4e06baac-07fc-4792-8ebe-158a470fd7b6, lookup_type: ById(4e06baac-07fc-4792-8ebe-158a470fd7b6) }, key: 754b28bf-6c2b-4d04-be10-941b3bfa03d8, lookup_type: ById(754b28bf-6c2b-4d04-be10-941b3bfa03d8) } result = Ok(()) saga_id = 8f55e372-61a7-41a4-a363-d8a319d45da5 saga_name = start-instance-update saga_node = FetchStateAndStartRealSaga
45872024-11-28T02:04:37.072ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 8f55e372-61a7-41a4-a363-d8a319d45da5 saga_name = start-instance-update saga_node = FetchStateAndStartRealSaga
45882024-11-28T02:04:37.072ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 8f55e372-61a7-41a4-a363-d8a319d45da5 saga_name = start-instance-update saga_node = FetchStateAndStartRealSaga
45892024-11-28T02:04:37.082ZINFOnexus (SagaExecutor): instance update: starting real update saga... current.active_vmm = Some(Vmm { id: 7001f3a3-3a22-4fd6-a06f-efde5d6ec855, time_created: 2024-11-28T02:04:16.124786Z, time_deleted: None, instance_id: 754b28bf-6c2b-4d04-be10-941b3bfa03d8, sled_id: 313847b3-d670-4167-9b09-9e7d3dfcb2a4, propolis_ip: V6(Ipv6Network { addr: fd00:1122:3344:101::1:0, prefix: 128 }), propolis_port: SqlU16(12400), runtime: VmmRuntimeState { time_state_updated: 2024-11-28T02:04:36.911491Z, gen: Generation(Generation(8)), state: Destroyed } }) current.migration = None current.runtime_state = InstanceRuntimeState { time_updated: 2024-11-28T02:04:14.018215Z, gen: Generation(Generation(3)), propolis_id: Some(7001f3a3-3a22-4fd6-a06f-efde5d6ec855), dst_propolis_id: None, migration_id: None, nexus_state: Vmm, time_last_auto_restarted: None } current.target_vmm = None file = nexus/src/app/sagas/instance_update/start.rs:180 instance_id = 754b28bf-6c2b-4d04-be10-941b3bfa03d8 saga_id = 8f55e372-61a7-41a4-a363-d8a319d45da5 saga_name = start-instance-update update.deprovision = true update.destroy_active_vmm = Some(7001f3a3-3a22-4fd6-a06f-efde5d6ec855 (propolis)) update.destroy_target_vmm = None update.network_config_update = Some(Delete) update.new_runtime_state = InstanceRuntimeState { time_updated: 2024-11-28T02:04:37.082616710Z, gen: Generation(Generation(4)), propolis_id: None, dst_propolis_id: None, migration_id: None, nexus_state: NoVmm, time_last_auto_restarted: None }
45902024-11-28T02:04:37.082ZINFOnexus (SagaExecutor): preparing saga file = nexus/src/app/saga.rs:255 saga_id = 623ae46b-4134-41b4-9292-b79e19ce7030 saga_name = instance-update
45912024-11-28T02:04:37.082ZINFOnexus (SEC): saga create dag = {"end_node":14,"graph":{"edge_property":"directed","edges":[[0,1,null],[1,2,null],[2,3,null],[3,4,null],[4,5,null],[5,6,null],[6,7,null],[7,8,null],[9,10,null],[8,9,null],[10,11,null],[11,12,null],[13,0,null],[12,14,null]],"node_holes":[],"nodes":[{"Action":{"action_name":"common.uuid_generate","label":"GenerateInstanceLockId","name":"saga_instance_lock_id"}},{"Action":{"action_name":"instance_update.become_updater","label":"BecomeUpdater","name":"updater_lock"}},{"Constant":{"name":"network_config_update","value":"Delete"}},{"Action":{"action_name":"instance_update.update_network_config","label":"UpdateNetworkConfig","name":"update_network_config"}},{"Action":{"action_name":"instance_update.release_virtual_provisioning","label":"ReleaseVirtualProvisioning","name":"release_virtual_provisioning"}},{"Action":{"action_name":"instance_update.unassign_oximeter_producer","label":"UnassignOximeterProducer","name":"unassign_oximeter_producer"}},{"Action":{"action_name":"instance_update.commit_instance_updates","label":"CommitInstanceUpdates","name":"commit_instance_updates"}},{"Constant":{"name":"destroy_active_vmm_params","value":{"instance_id":"754b28bf-6c2b-4d04-be10-941b3bfa03d8","serialized_authn":{"kind":{"Authenticated":[{"actor":{"UserBuiltin":{"user_builtin_id":"001de000-05e4-4000-8000-000000000002"}}},null]}},"vmm_id":"7001f3a3-3a22-4fd6-a06f-efde5d6ec855"}}},{"SubsagaStart":{"params_node_name":"destroy_active_vmm_params","saga_name":"destroy_active_vmm"}},{"Action":{"action_name":"destroy_vmm.release_sled_resources","label":"ReleaseSledResources","name":"release_sled_resources"}},{"Action":{"action_name":"destroy_vmm.mark_vmm_deleted","label":"MarkVmmDeleted","name":"mark_vmm_deleted"}},{"SubsagaEnd":{"name":"active_vmm_destroyed"}},{"Action":{"action_name":"instance_update.chain_successor_saga","label":"ChainSuccessorSaga","name":"chain_successor_saga"}},{"Start":{"params":{"authz_instance":{"key":"754b28bf-6c2b-4d04-be10-941b3bfa03d8","lookup_type":{"ById":"754b28bf-6c2b-4d04-be10-941b3bfa03d8"},"parent":{"key":"4e06baac-07fc-4792-8ebe-158a470fd7b6","lookup_type":{"ById":"4e06baac-07fc-4792-8ebe-158a470fd7b6"},"parent":{"key":"7e159834-997a-4c03-b9db-1e91d783221c","lookup_type":{"ById":"7e159834-997a-4c03-b9db-1e91d783221c"},"parent":null}}},"orig_lock":{"locked_gen":2,"updater_id":"3e9e8414-5ef7-4e8a-8104-2e7206cb01a3"},"serialized_authn":{"kind":{"Authenticated":[{"actor":{"UserBuiltin":{"user_builtin_id":"001de000-05e4-4000-8000-000000000002"}}},null]}},"update":{"deprovision":{"cpus_diff":2,"project_id":"4e06baac-07fc-4792-8ebe-158a470fd7b6","ram_diff":1073741824},"destroy_active_vmm":"7001f3a3-3a22-4fd6-a06f-efde5d6ec855","destroy_target_vmm":null,"network_config":"Delete","new_runtime":{"dst_propolis_id":null,"gen":4,"migration_id":null,"nexus_state":"NoVmm","propolis_id":null,"time_last_auto_restarted":null,"time_updated":"2024-11-28T02:04:37.082616710Z"}}}}},"End"]},"saga_name":"instance-update","start_node":13} file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/steno-0.4.1/src/sec.rs:1146 saga_id = 623ae46b-4134-41b4-9292-b79e19ce7030 saga_name = instance-update sec_id = 2469a748-bcd2-4e0d-99cc-f79def0452cc
45922024-11-28T02:04:37.082ZINFOnexus (SecStore): creating saga file = nexus/db-queries/src/db/sec_store.rs:50 saga_id = 623ae46b-4134-41b4-9292-b79e19ce7030 saga_name = instance-update
45932024-11-28T02:04:37.087ZINFOnexus (SagaExecutor): starting saga file = nexus/src/app/saga.rs:353 saga_id = 623ae46b-4134-41b4-9292-b79e19ce7030 saga_name = instance-update
45942024-11-28T02:04:37.087ZINFOnexus (SEC): saga start file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/steno-0.4.1/src/sec.rs:967 saga_id = 623ae46b-4134-41b4-9292-b79e19ce7030 saga_name = instance-update sec_id = 2469a748-bcd2-4e0d-99cc-f79def0452cc
45952024-11-28T02:04:37.087ZDEBGnexus (SEC): saga log event new_state = N013 started sec_id = 2469a748-bcd2-4e0d-99cc-f79def0452cc
45962024-11-28T02:04:37.087ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 13 saga_id = 623ae46b-4134-41b4-9292-b79e19ce7030
45972024-11-28T02:04:37.092ZDEBGnexus (SEC): saga log event new_state = N013 succeeded sec_id = 2469a748-bcd2-4e0d-99cc-f79def0452cc
45982024-11-28T02:04:37.092ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 13 saga_id = 623ae46b-4134-41b4-9292-b79e19ce7030
45992024-11-28T02:04:37.096ZDEBGnexus (SEC): saga log event new_state = N000 started sec_id = 2469a748-bcd2-4e0d-99cc-f79def0452cc
46002024-11-28T02:04:37.096ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 0 saga_id = 623ae46b-4134-41b4-9292-b79e19ce7030
46012024-11-28T02:04:37.101ZDEBGnexus (SEC): saga log event new_state = N000 succeeded sec_id = 2469a748-bcd2-4e0d-99cc-f79def0452cc
46022024-11-28T02:04:37.101ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(String("049b7a05-4f47-4e47-8902-c939be497d43")) node_id = 0 saga_id = 623ae46b-4134-41b4-9292-b79e19ce7030
46032024-11-28T02:04:37.108ZDEBGnexus (SEC): saga log event new_state = N001 started sec_id = 2469a748-bcd2-4e0d-99cc-f79def0452cc
46042024-11-28T02:04:37.108ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 1 saga_id = 623ae46b-4134-41b4-9292-b79e19ce7030
46052024-11-28T02:04:37.114ZDEBGnexus (SagaExecutor): instance update: trying to become instance updater... instance_id = 754b28bf-6c2b-4d04-be10-941b3bfa03d8 parent_lock = UpdaterLock { updater_id: 3e9e8414-5ef7-4e8a-8104-2e7206cb01a3, locked_gen: Generation(Generation(2)) } saga_id = 623ae46b-4134-41b4-9292-b79e19ce7030 saga_name = instance-update
46062024-11-28T02:04:37.114ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 623ae46b-4134-41b4-9292-b79e19ce7030 saga_name = instance-update saga_node = BecomeUpdater
46072024-11-28T02:04:37.114ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 623ae46b-4134-41b4-9292-b79e19ce7030 saga_name = instance-update saga_node = BecomeUpdater
46082024-11-28T02:04:37.124ZDEBGnexus (SagaExecutor): inherited lock from 3e9e8414-5ef7-4e8a-8104-2e7206cb01a3 to 049b7a05-4f47-4e47-8902-c939be497d43 actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true instance_id = 754b28bf-6c2b-4d04-be10-941b3bfa03d8 locked_gen = Generation(Generation(3)) parent_gen = Generation(Generation(2)) parent_id = 3e9e8414-5ef7-4e8a-8104-2e7206cb01a3 saga_id = 623ae46b-4134-41b4-9292-b79e19ce7030 saga_name = instance-update saga_node = BecomeUpdater updater_id = 049b7a05-4f47-4e47-8902-c939be497d43
46092024-11-28T02:04:37.124ZINFOnexus (SagaExecutor): instance_update: Now, I am become Updater, the destroyer of VMMs. file = nexus/src/app/sagas/instance_update/mod.rs:964 instance_id = 754b28bf-6c2b-4d04-be10-941b3bfa03d8 saga_id = 623ae46b-4134-41b4-9292-b79e19ce7030 saga_name = instance-update
46102024-11-28T02:04:37.124ZDEBGnexus (SEC): saga log event new_state = N001 succeeded sec_id = 2469a748-bcd2-4e0d-99cc-f79def0452cc
46112024-11-28T02:04:37.124ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Object {"locked_gen": Number(3), "updater_id": String("049b7a05-4f47-4e47-8902-c939be497d43")}) node_id = 1 saga_id = 623ae46b-4134-41b4-9292-b79e19ce7030
46122024-11-28T02:04:37.127ZDEBGnexus (SEC): saga log event new_state = N002 started sec_id = 2469a748-bcd2-4e0d-99cc-f79def0452cc
46132024-11-28T02:04:37.127ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 2 saga_id = 623ae46b-4134-41b4-9292-b79e19ce7030
46142024-11-28T02:04:37.131ZDEBGnexus (SEC): saga log event new_state = N002 succeeded sec_id = 2469a748-bcd2-4e0d-99cc-f79def0452cc
46152024-11-28T02:04:37.131ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(String("Delete")) node_id = 2 saga_id = 623ae46b-4134-41b4-9292-b79e19ce7030
46162024-11-28T02:04:37.134ZDEBGnexus (SEC): saga log event new_state = N003 started sec_id = 2469a748-bcd2-4e0d-99cc-f79def0452cc
46172024-11-28T02:04:37.134ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 3 saga_id = 623ae46b-4134-41b4-9292-b79e19ce7030
46182024-11-28T02:04:37.137ZINFOnexus (SagaExecutor): instance update: deleting network config file = nexus/src/app/sagas/instance_update/mod.rs:1017 instance_id = 754b28bf-6c2b-4d04-be10-941b3bfa03d8 saga_id = 623ae46b-4134-41b4-9292-b79e19ce7030 saga_name = instance-update
46192024-11-28T02:04:37.142ZINFOnexus: deleting instance dpd configuration file = nexus/src/app/instance_network.rs:548 instance_id = 754b28bf-6c2b-4d04-be10-941b3bfa03d8
46202024-11-28T02:04:37.144ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 623ae46b-4134-41b4-9292-b79e19ce7030 saga_name = instance-update saga_node = UpdateNetworkConfig
46212024-11-28T02:04:37.144ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 623ae46b-4134-41b4-9292-b79e19ce7030 saga_name = instance-update saga_node = UpdateNetworkConfig
46222024-11-28T02:04:37.147ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 623ae46b-4134-41b4-9292-b79e19ce7030 saga_name = instance-update saga_node = UpdateNetworkConfig
46232024-11-28T02:04:37.147ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 623ae46b-4134-41b4-9292-b79e19ce7030 saga_name = instance-update saga_node = UpdateNetworkConfig
46242024-11-28T02:04:37.149ZWARNnexus: no matching nat entries to soft delete file = nexus/src/app/instance_network.rs:711
46252024-11-28T02:04:37.149ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 623ae46b-4134-41b4-9292-b79e19ce7030 saga_name = instance-update saga_node = UpdateNetworkConfig
46262024-11-28T02:04:37.149ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 623ae46b-4134-41b4-9292-b79e19ce7030 saga_name = instance-update saga_node = UpdateNetworkConfig
46272024-11-28T02:04:37.151ZWARNnexus: no matching nat entries to soft delete file = nexus/src/app/instance_network.rs:711
46282024-11-28T02:04:37.151ZDEBGnexus (InstanceAllocator): roles roles = RoleSet { roles: {} }
46292024-11-28T02:04:37.151ZDEBGnexus (InstanceAllocator): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Database result = Ok(())
46302024-11-28T02:04:37.153ZDEBGnexus (InstanceAllocator): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "viewer")} }
46312024-11-28T02:04:37.154ZDEBGnexus (InstanceAllocator): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Fleet result = Ok(())
46322024-11-28T02:04:37.154ZDEBGnexus (InstanceAllocator): roles roles = RoleSet { roles: {} }
46332024-11-28T02:04:37.154ZDEBGnexus (InstanceAllocator): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Database result = Ok(())
46342024-11-28T02:04:37.158ZDEBGnexus: notifying dendrite of updates instance_id = Some(754b28bf-6c2b-4d04-be10-941b3bfa03d8 (instance)) switch = switch0
46352024-11-28T02:04:37.159ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-313847b3-d670-4167-9b09-9e7d3dfcb2a4.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-313847b3-d670-4167-9b09-9e7d3dfcb2a4.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1543, tv_nsec: 629393641 } })
46362024-11-28T02:04:37.159ZINFOnexus: Determining switch slots managed by switch zones file = nexus/src/app/mod.rs:1103
46372024-11-28T02:04:37.189ZINFOnexus: determining switch slot managed by dendrite zone file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
46382024-11-28T02:04:37.189ZDEBGnexus (MgsClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
46392024-11-28T02:04:37.190ZDEBGnexus (MgsClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "c7c04556-6dcc-4437-a15e-679fa1c1a671", "content-length": "26", "date": "Thu, 28 Nov 2024 02:04:36 GMT"} })
46402024-11-28T02:04:37.190ZINFOnexus: identified switch slot for dendrite zone file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
46412024-11-28T02:04:37.190ZINFOnexus: completed mapping dendrite zones to switch slots file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
46422024-11-28T02:04:37.222ZDEBGnexus (DpdClient): client request body = None method = POST uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/trigger
46432024-11-28T02:04:37.223ZDEBGnexus (DpdClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/trigger", status: 200, headers: {"content-type": "application/json", "x-request-id": "fcd9f20d-7a8b-4223-9516-1df3c416d526", "content-length": "4", "date": "Thu, 28 Nov 2024 02:04:36 GMT"} })
46442024-11-28T02:04:37.223ZDEBGnexus (SEC): saga log event new_state = N003 succeeded sec_id = 2469a748-bcd2-4e0d-99cc-f79def0452cc
46452024-11-28T02:04:37.223ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 3 saga_id = 623ae46b-4134-41b4-9292-b79e19ce7030
46462024-11-28T02:04:37.226ZDEBGnexus (SEC): saga log event new_state = N004 started sec_id = 2469a748-bcd2-4e0d-99cc-f79def0452cc
46472024-11-28T02:04:37.226ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 4 saga_id = 623ae46b-4134-41b4-9292-b79e19ce7030
46482024-11-28T02:04:37.234ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 623ae46b-4134-41b4-9292-b79e19ce7030 saga_name = instance-update saga_node = ReleaseVirtualProvisioning
46492024-11-28T02:04:37.234ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 623ae46b-4134-41b4-9292-b79e19ce7030 saga_name = instance-update saga_node = ReleaseVirtualProvisioning
46502024-11-28T02:04:37.248ZINFOnexus (SagaExecutor): instance update (no VMM): deallocated virtual provisioning resources file = nexus/src/app/sagas/instance_update/mod.rs:1092 instance_id = 754b28bf-6c2b-4d04-be10-941b3bfa03d8 records_deleted = [VirtualProvisioningCollection { id: 001de000-1334-4000-8000-000000000000, time_modified: Some(2024-11-28T02:04:37.237332Z), collection_type: "Fleet", virtual_disk_bytes_provisioned: ByteCount(ByteCount(6442450944)), cpus_provisioned: 0, ram_provisioned: ByteCount(ByteCount(0)) }, VirtualProvisioningCollection { id: 4e06baac-07fc-4792-8ebe-158a470fd7b6, time_modified: Some(2024-11-28T02:04:37.237332Z), collection_type: "Project", virtual_disk_bytes_provisioned: ByteCount(ByteCount(2147483648)), cpus_provisioned: 0, ram_provisioned: ByteCount(ByteCount(0)) }, VirtualProvisioningCollection { id: 7e159834-997a-4c03-b9db-1e91d783221c, time_modified: Some(2024-11-28T02:04:37.237332Z), collection_type: "Silo", virtual_disk_bytes_provisioned: ByteCount(ByteCount(6442450944)), cpus_provisioned: 0, ram_provisioned: ByteCount(ByteCount(0)) }] saga_id = 623ae46b-4134-41b4-9292-b79e19ce7030 saga_name = instance-update
46512024-11-28T02:04:37.248ZDEBGnexus (SEC): saga log event new_state = N004 succeeded sec_id = 2469a748-bcd2-4e0d-99cc-f79def0452cc
46522024-11-28T02:04:37.248ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 4 saga_id = 623ae46b-4134-41b4-9292-b79e19ce7030
46532024-11-28T02:04:37.251ZDEBGnexus (SEC): saga log event new_state = N005 started sec_id = 2469a748-bcd2-4e0d-99cc-f79def0452cc
46542024-11-28T02:04:37.251ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 5 saga_id = 623ae46b-4134-41b4-9292-b79e19ce7030
46552024-11-28T02:04:37.255ZINFOnexus (SagaExecutor): instance update (no VMM): unassigning oximeter producer file = nexus/src/app/sagas/instance_update/mod.rs:1127 instance_id = 754b28bf-6c2b-4d04-be10-941b3bfa03d8 saga_id = 623ae46b-4134-41b4-9292-b79e19ce7030 saga_name = instance-update
46562024-11-28T02:04:37.257ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 623ae46b-4134-41b4-9292-b79e19ce7030 saga_name = instance-update saga_node = UnassignOximeterProducer
46572024-11-28T02:04:37.257ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 623ae46b-4134-41b4-9292-b79e19ce7030 saga_name = instance-update saga_node = UnassignOximeterProducer
46582024-11-28T02:04:37.262ZDEBGnexus (SagaExecutor): deleted metric producer assignment collector_id = 119eed64-2e24-4d5f-8c42-be6cd8bec8b5 producer_id = 754b28bf-6c2b-4d04-be10-941b3bfa03d8 saga_id = 623ae46b-4134-41b4-9292-b79e19ce7030 saga_name = instance-update
46592024-11-28T02:04:37.262ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 623ae46b-4134-41b4-9292-b79e19ce7030 saga_name = instance-update saga_node = UnassignOximeterProducer
46602024-11-28T02:04:37.262ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 623ae46b-4134-41b4-9292-b79e19ce7030 saga_name = instance-update saga_node = UnassignOximeterProducer
46612024-11-28T02:04:37.298ZINFOnexus (SagaExecutor): registered oximeter collector client file = nexus/src/app/oximeter.rs:262 id = 754b28bf-6c2b-4d04-be10-941b3bfa03d8 saga_id = 623ae46b-4134-41b4-9292-b79e19ce7030 saga_name = instance-update
46622024-11-28T02:04:37.301ZDEBGnexus (SagaExecutor): client request body = None method = DELETE oximeter-collector = 754b28bf-6c2b-4d04-be10-941b3bfa03d8 saga_id = 623ae46b-4134-41b4-9292-b79e19ce7030 saga_name = instance-update uri = http://[fd00:1122:3344:101::d]:12223/producers/754b28bf-6c2b-4d04-be10-941b3bfa03d8
46632024-11-28T02:04:37.319ZDEBGnexus (SagaExecutor): client response oximeter-collector = 754b28bf-6c2b-4d04-be10-941b3bfa03d8 result = Ok(Response { url: "http://[fd00:1122:3344:101::d]:12223/producers/754b28bf-6c2b-4d04-be10-941b3bfa03d8", status: 204, headers: {"x-request-id": "18a3678e-2353-4397-a4ca-700505f9bbf4", "date": "Thu, 28 Nov 2024 02:04:37 GMT"} }) saga_id = 623ae46b-4134-41b4-9292-b79e19ce7030 saga_name = instance-update
46642024-11-28T02:04:37.319ZDEBGnexus (SagaExecutor): successfully deleted producer from collector address = [fd00:1122:3344:101::d]:12223 collector_id = 119eed64-2e24-4d5f-8c42-be6cd8bec8b5 producer_id = 754b28bf-6c2b-4d04-be10-941b3bfa03d8 saga_id = 623ae46b-4134-41b4-9292-b79e19ce7030 saga_name = instance-update
46652024-11-28T02:04:37.321ZDEBGnexus (SEC): saga log event new_state = N005 succeeded sec_id = 2469a748-bcd2-4e0d-99cc-f79def0452cc
46662024-11-28T02:04:37.321ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 5 saga_id = 623ae46b-4134-41b4-9292-b79e19ce7030
46672024-11-28T02:04:37.325ZDEBGnexus (SEC): saga log event new_state = N006 started sec_id = 2469a748-bcd2-4e0d-99cc-f79def0452cc
46682024-11-28T02:04:37.325ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 6 saga_id = 623ae46b-4134-41b4-9292-b79e19ce7030
46692024-11-28T02:04:37.329ZDEBGnexus (SagaExecutor): instance update: committing new runtime state and unlocking... instance_id = 754b28bf-6c2b-4d04-be10-941b3bfa03d8 lock = UpdaterLock { updater_id: 049b7a05-4f47-4e47-8902-c939be497d43, locked_gen: Generation(Generation(3)) } new_runtime = InstanceRuntimeState { time_updated: 2024-11-28T02:04:37.082616710Z, gen: Generation(Generation(4)), propolis_id: None, dst_propolis_id: None, migration_id: None, nexus_state: NoVmm, time_last_auto_restarted: None } saga_id = 623ae46b-4134-41b4-9292-b79e19ce7030 saga_name = instance-update
46702024-11-28T02:04:37.329ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 623ae46b-4134-41b4-9292-b79e19ce7030 saga_name = instance-update saga_node = CommitInstanceUpdates
46712024-11-28T02:04:37.329ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 623ae46b-4134-41b4-9292-b79e19ce7030 saga_name = instance-update saga_node = CommitInstanceUpdates
46722024-11-28T02:04:37.363ZINFOnexus (SagaExecutor): instance update: committed update new runtime state! did_unlock = true file = nexus/src/app/sagas/instance_update/mod.rs:1176 instance_id = 754b28bf-6c2b-4d04-be10-941b3bfa03d8 new_runtime = InstanceRuntimeState { time_updated: 2024-11-28T02:04:37.082616710Z, gen: Generation(Generation(4)), propolis_id: None, dst_propolis_id: None, migration_id: None, nexus_state: NoVmm, time_last_auto_restarted: None } saga_id = 623ae46b-4134-41b4-9292-b79e19ce7030 saga_name = instance-update
46732024-11-28T02:04:37.363ZDEBGnexus (SEC): saga log event new_state = N006 succeeded sec_id = 2469a748-bcd2-4e0d-99cc-f79def0452cc
46742024-11-28T02:04:37.363ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 6 saga_id = 623ae46b-4134-41b4-9292-b79e19ce7030
46752024-11-28T02:04:37.363ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 25 reason = Signaled
46762024-11-28T02:04:37.363ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 24 reason = Signaled
46772024-11-28T02:04:37.363ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
46782024-11-28T02:04:37.363ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
46792024-11-28T02:04:37.363ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
46802024-11-28T02:04:37.363ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
46812024-11-28T02:04:37.363ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
46822024-11-28T02:04:37.365ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
46832024-11-28T02:04:37.366ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
46842024-11-28T02:04:37.366ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
46852024-11-28T02:04:37.367ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
46862024-11-28T02:04:37.367ZDEBGnexus (SEC): saga log event new_state = N007 started sec_id = 2469a748-bcd2-4e0d-99cc-f79def0452cc
46872024-11-28T02:04:37.367ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 7 saga_id = 623ae46b-4134-41b4-9292-b79e19ce7030
46882024-11-28T02:04:37.368ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
46892024-11-28T02:04:37.370ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
46902024-11-28T02:04:37.370ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
46912024-11-28T02:04:37.370ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
46922024-11-28T02:04:37.371ZDEBGnexus (SEC): saga log event new_state = N007 succeeded sec_id = 2469a748-bcd2-4e0d-99cc-f79def0452cc
46932024-11-28T02:04:37.371ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Object {"instance_id": String("754b28bf-6c2b-4d04-be10-941b3bfa03d8"), "serialized_authn": Object {"kind": Object {"Authenticated": Array [Object {"actor": Object {"UserBuiltin": Object {"user_builtin_id": String("001de000-05e4-4000-8000-000000000002")}}}, Null]}}, "vmm_id": String("7001f3a3-3a22-4fd6-a06f-efde5d6ec855")}) node_id = 7 saga_id = 623ae46b-4134-41b4-9292-b79e19ce7030
46942024-11-28T02:04:37.375ZDEBGnexus (SEC): saga log event new_state = N008 started sec_id = 2469a748-bcd2-4e0d-99cc-f79def0452cc
46952024-11-28T02:04:37.375ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 8 saga_id = 623ae46b-4134-41b4-9292-b79e19ce7030
46962024-11-28T02:04:37.379ZDEBGnexus (SEC): saga log event new_state = N008 succeeded sec_id = 2469a748-bcd2-4e0d-99cc-f79def0452cc
46972024-11-28T02:04:37.379ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 8 saga_id = 623ae46b-4134-41b4-9292-b79e19ce7030
46982024-11-28T02:04:37.383ZDEBGnexus (SEC): saga log event new_state = N009 started sec_id = 2469a748-bcd2-4e0d-99cc-f79def0452cc
46992024-11-28T02:04:37.383ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 9 saga_id = 623ae46b-4134-41b4-9292-b79e19ce7030
47002024-11-28T02:04:37.386ZINFOnexus (SagaExecutor): instance update (VMM destroyed): deallocating sled resource reservation file = nexus/src/app/sagas/instance_update/destroyed.rs:83 instance_id = 754b28bf-6c2b-4d04-be10-941b3bfa03d8 propolis_id = 7001f3a3-3a22-4fd6-a06f-efde5d6ec855 saga_id = 623ae46b-4134-41b4-9292-b79e19ce7030 saga_name = instance-update
47012024-11-28T02:04:37.386ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 623ae46b-4134-41b4-9292-b79e19ce7030 saga_name = instance-update saga_node = ReleaseSledResources
47022024-11-28T02:04:37.386ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 623ae46b-4134-41b4-9292-b79e19ce7030 saga_name = instance-update saga_node = ReleaseSledResources
47032024-11-28T02:04:37.404ZINFOnexus (BackgroundTasks): VPC route manager sled 313847b3-d670-4167-9b09-9e7d3dfcb2a4 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
47042024-11-28T02:04:37.404ZDEBGnexus (BackgroundTasks): client request SledAgent = 313847b3-d670-4167-9b09-9e7d3dfcb2a4 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
47052024-11-28T02:04:37.404ZDEBGnexus (BackgroundTasks): client response SledAgent = 313847b3-d670-4167-9b09-9e7d3dfcb2a4 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "52ebc99a-ecd3-4086-bbf4-8f3ae2eddee0", "content-length": "372", "date": "Thu, 28 Nov 2024 02:04:37 GMT"} })
47062024-11-28T02:04:37.404ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
47072024-11-28T02:04:37.404ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
47082024-11-28T02:04:37.408ZDEBGnexus (SEC): saga log event new_state = N009 succeeded sec_id = 2469a748-bcd2-4e0d-99cc-f79def0452cc
47092024-11-28T02:04:37.408ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 9 saga_id = 623ae46b-4134-41b4-9292-b79e19ce7030
47102024-11-28T02:04:37.409ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = b4f66e8f-d26f-4bc5-b13d-6c1087e7b7cc roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
47112024-11-28T02:04:37.410ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = b4f66e8f-d26f-4bc5-b13d-6c1087e7b7cc resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
47122024-11-28T02:04:37.411ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1907 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = b4f66e8f-d26f-4bc5-b13d-6c1087e7b7cc response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
47132024-11-28T02:04:37.412ZDEBGnexus (SEC): saga log event new_state = N010 started sec_id = 2469a748-bcd2-4e0d-99cc-f79def0452cc
47142024-11-28T02:04:37.412ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 10 saga_id = 623ae46b-4134-41b4-9292-b79e19ce7030
47152024-11-28T02:04:37.416ZINFOnexus (SagaExecutor): instance update (VMM destroyed): marking VMM record deleted file = nexus/src/app/sagas/instance_update/destroyed.rs:114 instance_id = 754b28bf-6c2b-4d04-be10-941b3bfa03d8 propolis_id = 7001f3a3-3a22-4fd6-a06f-efde5d6ec855 saga_id = 623ae46b-4134-41b4-9292-b79e19ce7030 saga_name = instance-update
47162024-11-28T02:04:37.416ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 623ae46b-4134-41b4-9292-b79e19ce7030 saga_name = instance-update saga_node = MarkVmmDeleted
47172024-11-28T02:04:37.416ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 623ae46b-4134-41b4-9292-b79e19ce7030 saga_name = instance-update saga_node = MarkVmmDeleted
47182024-11-28T02:04:37.424ZDEBGnexus (SEC): saga log event new_state = N010 succeeded sec_id = 2469a748-bcd2-4e0d-99cc-f79def0452cc
47192024-11-28T02:04:37.424ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 10 saga_id = 623ae46b-4134-41b4-9292-b79e19ce7030
47202024-11-28T02:04:37.436ZDEBGnexus (SEC): saga log event new_state = N011 started sec_id = 2469a748-bcd2-4e0d-99cc-f79def0452cc
47212024-11-28T02:04:37.436ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 11 saga_id = 623ae46b-4134-41b4-9292-b79e19ce7030
47222024-11-28T02:04:37.444ZDEBGnexus (SEC): saga log event new_state = N011 succeeded sec_id = 2469a748-bcd2-4e0d-99cc-f79def0452cc
47232024-11-28T02:04:37.444ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 11 saga_id = 623ae46b-4134-41b4-9292-b79e19ce7030
47242024-11-28T02:04:37.452ZDEBGnexus (SEC): saga log event new_state = N012 started sec_id = 2469a748-bcd2-4e0d-99cc-f79def0452cc
47252024-11-28T02:04:37.452ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 12 saga_id = 623ae46b-4134-41b4-9292-b79e19ce7030
47262024-11-28T02:04:37.456ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 623ae46b-4134-41b4-9292-b79e19ce7030 saga_name = instance-update saga_node = ChainSuccessorSaga
47272024-11-28T02:04:37.456ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 623ae46b-4134-41b4-9292-b79e19ce7030 saga_name = instance-update saga_node = ChainSuccessorSaga
47282024-11-28T02:04:37.456ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {72e78d15-33bc-4519-9d15-3e4071491759: {10.151.2.175: {b37eaf51-00a7-4d94-9ed6-fae8cf836ab7}, 10.151.2.176: {b37eaf51-00a7-4d94-9ed6-fae8cf836ab7}}} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
47292024-11-28T02:04:37.456ZDEBGnexus (BackgroundTasks): client request SledAgent = 313847b3-d670-4167-9b09-9e7d3dfcb2a4 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
47302024-11-28T02:04:37.457ZDEBGnexus (BackgroundTasks): client response SledAgent = 313847b3-d670-4167-9b09-9e7d3dfcb2a4 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "3f358c6b-e78c-4d2d-a4af-531014954b86", "date": "Thu, 28 Nov 2024 02:04:37 GMT"} })
47312024-11-28T02:04:37.457ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
47322024-11-28T02:04:37.457ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
47332024-11-28T02:04:37.458ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 623ae46b-4134-41b4-9292-b79e19ce7030 saga_name = instance-update saga_node = ChainSuccessorSaga
47342024-11-28T02:04:37.458ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 623ae46b-4134-41b4-9292-b79e19ce7030 saga_name = instance-update saga_node = ChainSuccessorSaga
47352024-11-28T02:04:37.459ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
47362024-11-28T02:04:37.459ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 623ae46b-4134-41b4-9292-b79e19ce7030 saga_name = instance-update saga_node = ChainSuccessorSaga
47372024-11-28T02:04:37.459ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
47382024-11-28T02:04:37.459ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 623ae46b-4134-41b4-9292-b79e19ce7030 saga_name = instance-update saga_node = ChainSuccessorSaga
47392024-11-28T02:04:37.461ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } saga_id = 623ae46b-4134-41b4-9292-b79e19ce7030 saga_name = instance-update saga_node = ChainSuccessorSaga
47402024-11-28T02:04:37.462ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
47412024-11-28T02:04:37.462ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
47422024-11-28T02:04:37.463ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
47432024-11-28T02:04:37.464ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
47442024-11-28T02:04:37.464ZDEBGnexus (SagaExecutor): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Instance { parent: Project { parent: Silo { parent: Fleet, key: 7e159834-997a-4c03-b9db-1e91d783221c, lookup_type: ById(7e159834-997a-4c03-b9db-1e91d783221c) }, key: 4e06baac-07fc-4792-8ebe-158a470fd7b6, lookup_type: ById(4e06baac-07fc-4792-8ebe-158a470fd7b6) }, key: 754b28bf-6c2b-4d04-be10-941b3bfa03d8, lookup_type: ById(754b28bf-6c2b-4d04-be10-941b3bfa03d8) } result = Ok(()) saga_id = 623ae46b-4134-41b4-9292-b79e19ce7030 saga_name = instance-update saga_node = ChainSuccessorSaga
47452024-11-28T02:04:37.464ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 623ae46b-4134-41b4-9292-b79e19ce7030 saga_name = instance-update saga_node = ChainSuccessorSaga
47462024-11-28T02:04:37.464ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 623ae46b-4134-41b4-9292-b79e19ce7030 saga_name = instance-update saga_node = ChainSuccessorSaga
47472024-11-28T02:04:37.465ZINFOnexus (BackgroundTasks): VPC route manager sled 313847b3-d670-4167-9b09-9e7d3dfcb2a4 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
47482024-11-28T02:04:37.465ZDEBGnexus (BackgroundTasks): client request SledAgent = 313847b3-d670-4167-9b09-9e7d3dfcb2a4 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
47492024-11-28T02:04:37.465ZDEBGnexus (BackgroundTasks): client response SledAgent = 313847b3-d670-4167-9b09-9e7d3dfcb2a4 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "8f5bcf07-a794-4354-875f-fbe54a46f0d2", "date": "Thu, 28 Nov 2024 02:04:37 GMT"} })
47502024-11-28T02:04:37.467ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 104.364233ms iteration = 25 status = {}
47512024-11-28T02:04:37.467ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:40402 req_id = c3391b67-d84b-4d6e-aa1f-9ee371c6cb79 roles = RoleSet { roles: {} } uri = /nat/ipv4/changeset/10?limit=100
47522024-11-28T02:04:37.467ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:40402 req_id = c3391b67-d84b-4d6e-aa1f-9ee371c6cb79 resource = Database result = Ok(()) uri = /nat/ipv4/changeset/10?limit=100
47532024-11-28T02:04:37.470ZDEBGnexus (SagaExecutor): instance update: instance will not reincarnate auto_restart_config = InstanceAutoRestart { policy: None, cooldown: None } instance_id = 754b28bf-6c2b-4d04-be10-941b3bfa03d8 karmic_state = WillReincarnate needs_reincarnation = false saga_id = 623ae46b-4134-41b4-9292-b79e19ce7030 saga_name = instance-update
47542024-11-28T02:04:37.470ZDEBGnexus (SEC): saga log event new_state = N012 succeeded sec_id = 2469a748-bcd2-4e0d-99cc-f79def0452cc
47552024-11-28T02:04:37.470ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 12 saga_id = 623ae46b-4134-41b4-9292-b79e19ce7030
47562024-11-28T02:04:37.470ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2529 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:40402 req_id = c3391b67-d84b-4d6e-aa1f-9ee371c6cb79 response_code = 200 uri = /nat/ipv4/changeset/10?limit=100
47572024-11-28T02:04:37.473ZDEBGnexus (SEC): saga log event new_state = N014 started sec_id = 2469a748-bcd2-4e0d-99cc-f79def0452cc
47582024-11-28T02:04:37.473ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 14 saga_id = 623ae46b-4134-41b4-9292-b79e19ce7030
47592024-11-28T02:04:37.476ZDEBGnexus (SEC): saga log event new_state = N014 succeeded sec_id = 2469a748-bcd2-4e0d-99cc-f79def0452cc
47602024-11-28T02:04:37.476ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 14 saga_id = 623ae46b-4134-41b4-9292-b79e19ce7030
47612024-11-28T02:04:37.480ZINFOnexus (SEC): update for saga cached state file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/steno-0.4.1/src/sec.rs:1384 new_state = Done saga_id = 623ae46b-4134-41b4-9292-b79e19ce7030 sec_id = 2469a748-bcd2-4e0d-99cc-f79def0452cc
47622024-11-28T02:04:37.480ZINFOnexus (SecStore): updating state file = nexus/db-queries/src/db/sec_store.rs:95 new_state = done saga_id = 623ae46b-4134-41b4-9292-b79e19ce7030
47632024-11-28T02:04:37.513ZINFOnexus (SEC): saga finished file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/steno-0.4.1/src/sec.rs:1025 result = success saga_id = 623ae46b-4134-41b4-9292-b79e19ce7030 saga_name = instance-update sec_id = 2469a748-bcd2-4e0d-99cc-f79def0452cc
47642024-11-28T02:04:37.513ZINFOnexus (SagaExecutor): saga finished file = nexus/src/app/saga.rs:404 saga_id = 623ae46b-4134-41b4-9292-b79e19ce7030 saga_name = instance-update saga_result = SagaResult { saga_id: 623ae46b-4134-41b4-9292-b79e19ce7030, saga_log: SagaLog { saga_id: 623ae46b-4134-41b4-9292-b79e19ce7030, unwinding: false, events: [N013 started, N013 succeeded, N000 started, N000 succeeded, N001 started, N001 succeeded, N002 started, N002 succeeded, N003 started, N003 succeeded, N004 started, N004 succeeded, N005 started, N005 succeeded, N006 started, N006 succeeded, N007 started, N007 succeeded, N008 started, N008 succeeded, N009 started, N009 succeeded, N010 started, N010 succeeded, N011 started, N011 succeeded, N012 started, N012 succeeded, N014 started, N014 succeeded], node_status: {0: Succeeded(String("049b7a05-4f47-4e47-8902-c939be497d43")), 1: Succeeded(Object {"locked_gen": Number(3), "updater_id": String("049b7a05-4f47-4e47-8902-c939be497d43")}), 2: Succeeded(String("Delete")), 3: Succeeded(Null), 4: Succeeded(Null), 5: Succeeded(Null), 6: Succeeded(Null), 7: Succeeded(Object {"instance_id": String("754b28bf-6c2b-4d04-be10-941b3bfa03d8"), "serialized_authn": Object {"kind": Object {"Authenticated": Array [Object {"actor": Object {"UserBuiltin": Object {"user_builtin_id": String("001de000-05e4-4000-8000-000000000002")}}}, Null]}}, "vmm_id": String("7001f3a3-3a22-4fd6-a06f-efde5d6ec855")}), 8: Succeeded(Null), 9: Succeeded(Null), 10: Succeeded(Null), 11: Succeeded(Null), 12: Succeeded(Null), 13: Succeeded(Null), 14: Succeeded(Null)} }, kind: Ok(SagaResultOk { saga_output: Null, node_outputs: {"active_vmm_destroyed": Null, "chain_successor_saga": Null, "commit_instance_updates": Null, "destroy_active_vmm_params": Object {"instance_id": String("754b28bf-6c2b-4d04-be10-941b3bfa03d8"), "serialized_authn": Object {"kind": Object {"Authenticated": Array [Object {"actor": Object {"UserBuiltin": Object {"user_builtin_id": String("001de000-05e4-4000-8000-000000000002")}}}, Null]}}, "vmm_id": String("7001f3a3-3a22-4fd6-a06f-efde5d6ec855")}, "mark_vmm_deleted": Null, "network_config_update": String("Delete"), "release_sled_resources": Null, "release_virtual_provisioning": Null, "saga_instance_lock_id": String("049b7a05-4f47-4e47-8902-c939be497d43"), "unassign_oximeter_producer": Null, "update_network_config": Null, "updater_lock": Object {"locked_gen": Number(3), "updater_id": String("049b7a05-4f47-4e47-8902-c939be497d43")}} }) }
47652024-11-28T02:04:37.513ZDEBGnexus (SagaExecutor): instance update: child saga completed successfully child_saga_id = 623ae46b-4134-41b4-9292-b79e19ce7030 instance_id = 754b28bf-6c2b-4d04-be10-941b3bfa03d8 saga_id = 8f55e372-61a7-41a4-a363-d8a319d45da5 saga_name = start-instance-update
47662024-11-28T02:04:37.513ZDEBGnexus (SEC): saga log event new_state = N002 succeeded sec_id = 2469a748-bcd2-4e0d-99cc-f79def0452cc
47672024-11-28T02:04:37.513ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 2 saga_id = 8f55e372-61a7-41a4-a363-d8a319d45da5
47682024-11-28T02:04:37.517ZDEBGnexus (SEC): saga log event new_state = N004 started sec_id = 2469a748-bcd2-4e0d-99cc-f79def0452cc
47692024-11-28T02:04:37.517ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 4 saga_id = 8f55e372-61a7-41a4-a363-d8a319d45da5
47702024-11-28T02:04:37.521ZDEBGnexus (SEC): saga log event new_state = N004 succeeded sec_id = 2469a748-bcd2-4e0d-99cc-f79def0452cc
47712024-11-28T02:04:37.521ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 4 saga_id = 8f55e372-61a7-41a4-a363-d8a319d45da5
47722024-11-28T02:04:37.524ZINFOnexus (SEC): update for saga cached state file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/steno-0.4.1/src/sec.rs:1384 new_state = Done saga_id = 8f55e372-61a7-41a4-a363-d8a319d45da5 sec_id = 2469a748-bcd2-4e0d-99cc-f79def0452cc
47732024-11-28T02:04:37.524ZINFOnexus (SecStore): updating state file = nexus/db-queries/src/db/sec_store.rs:95 new_state = done saga_id = 8f55e372-61a7-41a4-a363-d8a319d45da5
47742024-11-28T02:04:37.524ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
47752024-11-28T02:04:37.524ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
47762024-11-28T02:04:37.526ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
47772024-11-28T02:04:37.528ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
47782024-11-28T02:04:37.528ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
47792024-11-28T02:04:37.528ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
47802024-11-28T02:04:37.529ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
47812024-11-28T02:04:37.531ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
47822024-11-28T02:04:37.531ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
47832024-11-28T02:04:37.531ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
47842024-11-28T02:04:37.555ZINFOnexus (SEC): saga finished file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/steno-0.4.1/src/sec.rs:1025 result = success saga_id = 8f55e372-61a7-41a4-a363-d8a319d45da5 saga_name = start-instance-update sec_id = 2469a748-bcd2-4e0d-99cc-f79def0452cc
47852024-11-28T02:04:37.555ZINFOnexus (SagaExecutor): saga finished file = nexus/src/app/saga.rs:404 saga_id = 8f55e372-61a7-41a4-a363-d8a319d45da5 saga_name = start-instance-update saga_result = SagaResult { saga_id: 8f55e372-61a7-41a4-a363-d8a319d45da5, saga_log: SagaLog { saga_id: 8f55e372-61a7-41a4-a363-d8a319d45da5, unwinding: false, events: [N003 started, N003 succeeded, N000 started, N000 succeeded, N001 started, N001 succeeded, N002 started, N002 succeeded, N004 started, N004 succeeded], node_status: {0: Succeeded(String("3e9e8414-5ef7-4e8a-8104-2e7206cb01a3")), 1: Succeeded(Object {"locked_gen": Number(2), "updater_id": String("3e9e8414-5ef7-4e8a-8104-2e7206cb01a3")}), 2: Succeeded(Null), 3: Succeeded(Null), 4: Succeeded(Null)} }, kind: Ok(SagaResultOk { saga_output: Null, node_outputs: {"saga_instance_lock_id": String("3e9e8414-5ef7-4e8a-8104-2e7206cb01a3"), "state": Null, "updater_lock": Object {"locked_gen": Number(2), "updater_id": String("3e9e8414-5ef7-4e8a-8104-2e7206cb01a3")}} }) }
47862024-11-28T02:04:37.616ZDEBGnexus (BackgroundTasks): client request SledAgent = 313847b3-d670-4167-9b09-9e7d3dfcb2a4 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
47872024-11-28T02:04:37.617ZDEBGnexus (BackgroundTasks): client response SledAgent = 313847b3-d670-4167-9b09-9e7d3dfcb2a4 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "44cc127f-b23d-456b-9b55-2ccce2a21812", "content-length": "691", "date": "Thu, 28 Nov 2024 02:04:37 GMT"} })
47882024-11-28T02:04:37.617ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
47892024-11-28T02:04:37.617ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
47902024-11-28T02:04:37.617ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
47912024-11-28T02:04:37.618ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 254.934318ms iteration = 24 status = {}
47922024-11-28T02:04:38.435ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = fc0a40c2-e727-45b1-ab49-cf60815b6516 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
47932024-11-28T02:04:38.436ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = fc0a40c2-e727-45b1-ab49-cf60815b6516 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
47942024-11-28T02:04:38.437ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1726 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = fc0a40c2-e727-45b1-ab49-cf60815b6516 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
47952024-11-28T02:04:39.186ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:55495 req_id = 57ec3395-3cf0-4dd9-a206-7655a5913460 roles = RoleSet { roles: {} } uri = /metrics/collectors/119eed64-2e24-4d5f-8c42-be6cd8bec8b5/producers?sort_by=id_ascending
47962024-11-28T02:04:39.186ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:55495 req_id = 57ec3395-3cf0-4dd9-a206-7655a5913460 resource = Database result = Ok(()) uri = /metrics/collectors/119eed64-2e24-4d5f-8c42-be6cd8bec8b5/producers?sort_by=id_ascending
47972024-11-28T02:04:39.188ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1776 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:55495 req_id = 57ec3395-3cf0-4dd9-a206-7655a5913460 response_code = 200 uri = /metrics/collectors/119eed64-2e24-4d5f-8c42-be6cd8bec8b5/producers?sort_by=id_ascending
47982024-11-28T02:04:39.188ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:55495 req_id = 8c2b645a-0154-42c0-acd3-abc169e88567 roles = RoleSet { roles: {} } uri = /metrics/collectors/119eed64-2e24-4d5f-8c42-be6cd8bec8b5/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJjNDM0YjQ0OC01MTk5LTQ2NmUtYWYzMi00ZjYyOWUzMWM3YjEifX0%3D
47992024-11-28T02:04:39.188ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:55495 req_id = 8c2b645a-0154-42c0-acd3-abc169e88567 resource = Database result = Ok(()) uri = /metrics/collectors/119eed64-2e24-4d5f-8c42-be6cd8bec8b5/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJjNDM0YjQ0OC01MTk5LTQ2NmUtYWYzMi00ZjYyOWUzMWM3YjEifX0%3D
48002024-11-28T02:04:39.190ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2108 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:55495 req_id = 8c2b645a-0154-42c0-acd3-abc169e88567 response_code = 200 uri = /metrics/collectors/119eed64-2e24-4d5f-8c42-be6cd8bec8b5/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJjNDM0YjQ0OC01MTk5LTQ2NmUtYWYzMi00ZjYyOWUzMWM3YjEifX0%3D
48012024-11-28T02:04:39.459ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = f13666c5-f862-4f79-bc0d-d39eaf8c0a64 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
48022024-11-28T02:04:39.460ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = f13666c5-f862-4f79-bc0d-d39eaf8c0a64 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
48032024-11-28T02:04:39.461ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1881 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = f13666c5-f862-4f79-bc0d-d39eaf8c0a64 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
48042024-11-28T02:04:39.472ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:40402 req_id = 770fd698-b7fb-4fe5-911f-191c7257334a roles = RoleSet { roles: {} } uri = /nat/ipv4/changeset/10?limit=100
48052024-11-28T02:04:39.473ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:40402 req_id = 770fd698-b7fb-4fe5-911f-191c7257334a resource = Database result = Ok(()) uri = /nat/ipv4/changeset/10?limit=100
48062024-11-28T02:04:39.475ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2148 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:40402 req_id = 770fd698-b7fb-4fe5-911f-191c7257334a response_code = 200 uri = /nat/ipv4/changeset/10?limit=100
48072024-11-28T02:04:40.479ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 70 reason = Timeout
48082024-11-28T02:04:40.479ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
48092024-11-28T02:04:40.479ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 24 reason = Timeout
48102024-11-28T02:04:40.479ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 25 reason = Timeout
48112024-11-28T02:04:40.479ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
48122024-11-28T02:04:40.479ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 24 reason = Timeout
48132024-11-28T02:04:40.479ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
48142024-11-28T02:04:40.479ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
48152024-11-28T02:04:40.479ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 24 reason = Timeout
48162024-11-28T02:04:40.479ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
48172024-11-28T02:04:40.479ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 26 reason = Timeout
48182024-11-28T02:04:40.479ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
48192024-11-28T02:04:40.479ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 24 reason = Timeout
48202024-11-28T02:04:40.479ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
48212024-11-28T02:04:40.479ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 24 reason = Timeout
48222024-11-28T02:04:40.479ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
48232024-11-28T02:04:40.479ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
48242024-11-28T02:04:40.479ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 70 reason = Timeout
48252024-11-28T02:04:40.479ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 24 reason = Timeout
48262024-11-28T02:04:40.479ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
48272024-11-28T02:04:40.479ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
48282024-11-28T02:04:40.479ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
48292024-11-28T02:04:40.479ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 24 reason = Timeout
48302024-11-28T02:04:40.479ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
48312024-11-28T02:04:40.479ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 24 reason = Timeout
48322024-11-28T02:04:40.479ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
48332024-11-28T02:04:40.479ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 28 reason = Timeout
48342024-11-28T02:04:40.479ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
48352024-11-28T02:04:40.479ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 24 reason = Timeout
48362024-11-28T02:04:40.479ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 24 reason = Timeout
48372024-11-28T02:04:40.479ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
48382024-11-28T02:04:40.479ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 24 reason = Timeout
48392024-11-28T02:04:40.479ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 24 reason = Timeout
48402024-11-28T02:04:40.479ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
48412024-11-28T02:04:40.479ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
48422024-11-28T02:04:40.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
48432024-11-28T02:04:40.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
48442024-11-28T02:04:40.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
48452024-11-28T02:04:40.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
48462024-11-28T02:04:40.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
48472024-11-28T02:04:40.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
48482024-11-28T02:04:40.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
48492024-11-28T02:04:40.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
48502024-11-28T02:04:40.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
48512024-11-28T02:04:40.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
48522024-11-28T02:04:40.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
48532024-11-28T02:04:40.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
48542024-11-28T02:04:40.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
48552024-11-28T02:04:40.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
48562024-11-28T02:04:40.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
48572024-11-28T02:04:40.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
48582024-11-28T02:04:40.482ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
48592024-11-28T02:04:40.482ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
48602024-11-28T02:04:40.483ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 97600c65-7a6a-47c6-af0c-e62587c6d663 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
48612024-11-28T02:04:40.483ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 4.378726ms iteration = 24 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
48622024-11-28T02:04:40.483ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 4.586939ms iteration = 24 status = {"errors":[],"garbage_collect_requested":[]}
48632024-11-28T02:04:40.483ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
48642024-11-28T02:04:40.484ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 97600c65-7a6a-47c6-af0c-e62587c6d663 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
48652024-11-28T02:04:40.484ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
48662024-11-28T02:04:40.484ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
48672024-11-28T02:04:40.485ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
48682024-11-28T02:04:40.485ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
48692024-11-28T02:04:40.486ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-313847b3-d670-4167-9b09-9e7d3dfcb2a4.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-313847b3-d670-4167-9b09-9e7d3dfcb2a4.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1546, tv_nsec: 956190653 } })
48702024-11-28T02:04:40.486ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1103
48712024-11-28T02:04:40.486ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
48722024-11-28T02:04:40.486ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
48732024-11-28T02:04:40.486ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
48742024-11-28T02:04:40.487ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
48752024-11-28T02:04:40.487ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
48762024-11-28T02:04:40.487ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
48772024-11-28T02:04:40.488ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
48782024-11-28T02:04:40.488ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
48792024-11-28T02:04:40.488ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
48802024-11-28T02:04:40.490ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
48812024-11-28T02:04:40.490ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
48822024-11-28T02:04:40.490ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-313847b3-d670-4167-9b09-9e7d3dfcb2a4.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-313847b3-d670-4167-9b09-9e7d3dfcb2a4.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1546, tv_nsec: 960961026 } })
48832024-11-28T02:04:40.491ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1103
48842024-11-28T02:04:40.491ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
48852024-11-28T02:04:40.491ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 12.597649ms iteration = 24 status = {"errors":[],"records_set_to_done":[]}
48862024-11-28T02:04:40.492ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
48872024-11-28T02:04:40.492ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
48882024-11-28T02:04:40.492ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
48892024-11-28T02:04:40.492ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
48902024-11-28T02:04:40.493ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
48912024-11-28T02:04:40.493ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
48922024-11-28T02:04:40.493ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
48932024-11-28T02:04:40.493ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
48942024-11-28T02:04:40.493ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
48952024-11-28T02:04:40.493ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
48962024-11-28T02:04:40.494ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
48972024-11-28T02:04:40.494ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
48982024-11-28T02:04:40.495ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
48992024-11-28T02:04:40.495ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 12160 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 97600c65-7a6a-47c6-af0c-e62587c6d663 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
49002024-11-28T02:04:40.495ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
49012024-11-28T02:04:40.496ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
49022024-11-28T02:04:40.498ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-313847b3-d670-4167-9b09-9e7d3dfcb2a4.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-313847b3-d670-4167-9b09-9e7d3dfcb2a4.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1546, tv_nsec: 968105900 } })
49032024-11-28T02:04:40.498ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
49042024-11-28T02:04:40.498ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1103 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc
49052024-11-28T02:04:40.498ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
49062024-11-28T02:04:40.498ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
49072024-11-28T02:04:40.499ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
49082024-11-28T02:04:40.499ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
49092024-11-28T02:04:40.499ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
49102024-11-28T02:04:40.500ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
49112024-11-28T02:04:40.501ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
49122024-11-28T02:04:40.501ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
49132024-11-28T02:04:40.501ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
49142024-11-28T02:04:40.503ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
49152024-11-28T02:04:40.503ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
49162024-11-28T02:04:40.503ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 24.659928ms iteration = 24 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
49172024-11-28T02:04:40.507ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 67ac247f-c934-4c78-857a-fba4fbeb9937, lookup_type: ById(67ac247f-c934-4c78-857a-fba4fbeb9937) } result = Ok(())
49182024-11-28T02:04:40.507ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
49192024-11-28T02:04:40.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
49202024-11-28T02:04:40.523ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
49212024-11-28T02:04:40.523ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 67ac247f-c934-4c78-857a-fba4fbeb9937 original_time_created = 2024-11-28 01:53:03.159475 UTC target_id = 67ac247f-c934-4c78-857a-fba4fbeb9937 time_created = 2024-11-28 01:53:03.159475 UTC
49222024-11-28T02:04:40.523ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 44.574054ms iteration = 70 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"67ac247f-c934-4c78-857a-fba4fbeb9937","time_created":"2024-11-28T01:53:03.159475Z"}
49232024-11-28T02:04:40.524ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
49242024-11-28T02:04:40.525ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 46.491678ms iteration = 24 status = {"errors":[],"requests_created_ok":[],"start_invoked_ok":[]}
49252024-11-28T02:04:40.527ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
49262024-11-28T02:04:40.527ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
49272024-11-28T02:04:40.529ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
49282024-11-28T02:04:40.529ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
49292024-11-28T02:04:40.532ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
49302024-11-28T02:04:40.532ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
49312024-11-28T02:04:40.532ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
49322024-11-28T02:04:40.533ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 54.034789ms iteration = 70 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
49332024-11-28T02:04:40.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
49342024-11-28T02:04:40.533ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 54.563657ms iteration = 24 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[]}
49352024-11-28T02:04:40.539ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
49362024-11-28T02:04:40.541ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
49372024-11-28T02:04:40.541ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
49382024-11-28T02:04:40.541ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
49392024-11-28T02:04:40.562ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
49402024-11-28T02:04:40.562ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
49412024-11-28T02:04:40.568ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
49422024-11-28T02:04:40.568ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
49432024-11-28T02:04:40.568ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "c747a7a4-52ad-468f-879a-7625d5ac11ec", "content-length": "26", "date": "Thu, 28 Nov 2024 02:04:40 GMT"} })
49442024-11-28T02:04:40.568ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
49452024-11-28T02:04:40.570ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
49462024-11-28T02:04:40.578ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
49472024-11-28T02:04:40.578ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
49482024-11-28T02:04:40.578ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "3d796cce-e3ae-40cd-a56b-8a9e89dbe36c", "content-length": "26", "date": "Thu, 28 Nov 2024 02:04:40 GMT"} })
49492024-11-28T02:04:40.578ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
49502024-11-28T02:04:40.581ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
49512024-11-28T02:04:40.585ZINFOnexus (BackgroundTasks): VPC route manager sled 313847b3-d670-4167-9b09-9e7d3dfcb2a4 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
49522024-11-28T02:04:40.585ZDEBGnexus (BackgroundTasks): client request SledAgent = 313847b3-d670-4167-9b09-9e7d3dfcb2a4 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
49532024-11-28T02:04:40.585ZDEBGnexus (BackgroundTasks): client response SledAgent = 313847b3-d670-4167-9b09-9e7d3dfcb2a4 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "d6ff508e-f11c-4a9b-a9db-5c1cf98d4c21", "content-length": "372", "date": "Thu, 28 Nov 2024 02:04:40 GMT"} })
49542024-11-28T02:04:40.585ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
49552024-11-28T02:04:40.585ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
49562024-11-28T02:04:40.586ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1111 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc zone_address = fd00:1122:3344:101::2
49572024-11-28T02:04:40.586ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
49582024-11-28T02:04:40.586ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "6fc5914c-4230-47c6-b6b1-de0e66888d23", "content-length": "26", "date": "Thu, 28 Nov 2024 02:04:40 GMT"} })
49592024-11-28T02:04:40.586ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1116 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
49602024-11-28T02:04:40.587ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc
49612024-11-28T02:04:40.588ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
49622024-11-28T02:04:40.588ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
49632024-11-28T02:04:40.590ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
49642024-11-28T02:04:40.592ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 313847b3-d670-4167-9b09-9e7d3dfcb2a4, lookup_type: ById(313847b3-d670-4167-9b09-9e7d3dfcb2a4) } result = Ok(())
49652024-11-28T02:04:40.592ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
49662024-11-28T02:04:40.593ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
49672024-11-28T02:04:40.594ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 115.68305ms iteration = 24 status = 0
49682024-11-28T02:04:40.627ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 147.821238ms iteration = 24 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
49692024-11-28T02:04:40.631ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
49702024-11-28T02:04:40.631ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
49712024-11-28T02:04:40.631ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 152.532489ms iteration = 24 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
49722024-11-28T02:04:40.633ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
49732024-11-28T02:04:40.633ZDEBGnexus (BackgroundTasks): client request SledAgent = 313847b3-d670-4167-9b09-9e7d3dfcb2a4 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
49742024-11-28T02:04:40.633ZDEBGnexus (BackgroundTasks): client response SledAgent = 313847b3-d670-4167-9b09-9e7d3dfcb2a4 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "7dfb4e69-db67-4f3d-90fb-152df2d62331", "date": "Thu, 28 Nov 2024 02:04:40 GMT"} })
49752024-11-28T02:04:40.633ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
49762024-11-28T02:04:40.634ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
49772024-11-28T02:04:40.636ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
49782024-11-28T02:04:40.637ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
49792024-11-28T02:04:40.640ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
49802024-11-28T02:04:40.641ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
49812024-11-28T02:04:40.645ZINFOnexus (BackgroundTasks): VPC route manager sled 313847b3-d670-4167-9b09-9e7d3dfcb2a4 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
49822024-11-28T02:04:40.645ZDEBGnexus (BackgroundTasks): client request SledAgent = 313847b3-d670-4167-9b09-9e7d3dfcb2a4 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
49832024-11-28T02:04:40.645ZDEBGnexus (BackgroundTasks): client response SledAgent = 313847b3-d670-4167-9b09-9e7d3dfcb2a4 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "5fe00f4f-8bcd-4a84-8c86-ff3a6586f929", "date": "Thu, 28 Nov 2024 02:04:40 GMT"} })
49842024-11-28T02:04:40.646ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
49852024-11-28T02:04:40.646ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "4f4fb6a7-0f69-4f9f-9da6-71086232f5ea", "content-length": "2", "date": "Thu, 28 Nov 2024 02:04:40 GMT"} })
49862024-11-28T02:04:40.647ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 168.297603ms iteration = 26 status = {}
49872024-11-28T02:04:40.648ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
49882024-11-28T02:04:40.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
49892024-11-28T02:04:40.650ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 170.898598ms iteration = 24 status = null
49902024-11-28T02:04:40.667ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 188.326731ms iteration = 24 status = {}
49912024-11-28T02:04:40.699ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 220.355627ms iteration = 28 status = {"physical_disks_added":0}
49922024-11-28T02:04:40.717ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
49932024-11-28T02:04:40.717ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
49942024-11-28T02:04:40.719ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
49952024-11-28T02:04:40.721ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
49962024-11-28T02:04:40.721ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
49972024-11-28T02:04:40.721ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
49982024-11-28T02:04:40.726ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
49992024-11-28T02:04:40.729ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
50002024-11-28T02:04:40.729ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
50012024-11-28T02:04:40.730ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
50022024-11-28T02:04:40.781ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
50032024-11-28T02:04:40.781ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
50042024-11-28T02:04:40.783ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 4bb2986f-2e92-4ac3-9dc1-591b6bb60b3e, rack_id: 92a8ff6e-21d3-4c28-bcad-e323847985dc, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(ea300318-9d62-42ed-80ea-21c86590a166) } rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc switch_location = Switch0
50052024-11-28T02:04:40.783ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
50062024-11-28T02:04:40.784ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
50072024-11-28T02:04:40.789ZDEBGnexus (BackgroundTasks): client request SledAgent = 313847b3-d670-4167-9b09-9e7d3dfcb2a4 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
50082024-11-28T02:04:40.794ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
50092024-11-28T02:04:40.794ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "b1691b05-518d-466c-8573-fa6872407441", "content-length": "134", "date": "Thu, 28 Nov 2024 02:04:40 GMT"} })
50102024-11-28T02:04:40.794ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1746 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
50112024-11-28T02:04:40.794ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1773 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = ea300318-9d62-42ed-80ea-21c86590a166
50122024-11-28T02:04:40.794ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
50132024-11-28T02:04:40.794ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
50142024-11-28T02:04:40.794ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
50152024-11-28T02:04:40.794ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
50162024-11-28T02:04:40.794ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
50172024-11-28T02:04:40.794ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1920 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
50182024-11-28T02:04:40.794ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
50192024-11-28T02:04:40.794ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
50202024-11-28T02:04:40.795ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc
50212024-11-28T02:04:40.795ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
50222024-11-28T02:04:40.795ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
50232024-11-28T02:04:40.797ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
50242024-11-28T02:04:40.797ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "d151114e-07a4-4737-9854-c6dcf790cfd7", "content-length": "2", "date": "Thu, 28 Nov 2024 02:04:40 GMT"} })
50252024-11-28T02:04:40.797ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
50262024-11-28T02:04:40.797ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "b289008b-fb2a-4295-936e-283f61b8d39e", "content-length": "52", "date": "Thu, 28 Nov 2024 02:04:40 GMT"} })
50272024-11-28T02:04:40.797ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc switch_location = Switch0
50282024-11-28T02:04:40.797ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
50292024-11-28T02:04:40.870ZDEBGnexus (BackgroundTasks): client response SledAgent = 313847b3-d670-4167-9b09-9e7d3dfcb2a4 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "53d2d308-d982-4ab2-b5cc-64f4c7a6c64b", "content-length": "691", "date": "Thu, 28 Nov 2024 02:04:40 GMT"} })
50302024-11-28T02:04:40.871ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
50312024-11-28T02:04:40.871ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
50322024-11-28T02:04:40.871ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
50332024-11-28T02:04:40.873ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 394.230018ms iteration = 25 status = {}
50342024-11-28T02:04:40.964ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "3545b58f-b0c4-4a80-af45-6848cd668c0b", "date": "Thu, 28 Nov 2024 02:04:40 GMT"} })
50352024-11-28T02:04:40.964ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc switch = Switch0
50362024-11-28T02:04:40.964ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
50372024-11-28T02:04:40.965ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "0a9d653e-182f-4da3-b01d-974bb75a9b85", "content-length": "682", "date": "Thu, 28 Nov 2024 02:04:40 GMT"} })
50382024-11-28T02:04:40.965ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
50392024-11-28T02:04:40.965ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
50402024-11-28T02:04:40.970ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
50412024-11-28T02:04:40.970ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
50422024-11-28T02:04:40.974ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
50432024-11-28T02:04:40.974ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
50442024-11-28T02:04:40.977ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
50452024-11-28T02:04:40.977ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
50462024-11-28T02:04:40.985ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 506.305854ms iteration = 24 status = {}
50472024-11-28T02:04:41.523ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = bcffc8ea-4b8b-4d82-9df3-8e8a20a1e265 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
50482024-11-28T02:04:41.524ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = bcffc8ea-4b8b-4d82-9df3-8e8a20a1e265 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
50492024-11-28T02:04:41.526ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2619 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = bcffc8ea-4b8b-4d82-9df3-8e8a20a1e265 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
50502024-11-28T02:04:42.545ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 9c4c6e30-69d8-40ac-a5ec-04008ee92a30 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
50512024-11-28T02:04:42.546ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 9c4c6e30-69d8-40ac-a5ec-04008ee92a30 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
50522024-11-28T02:04:42.548ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2920 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 9c4c6e30-69d8-40ac-a5ec-04008ee92a30 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
50532024-11-28T02:04:43.566ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = ccb33b10-8789-4bda-b261-227a160f1a46 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
50542024-11-28T02:04:43.566ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = ccb33b10-8789-4bda-b261-227a160f1a46 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
50552024-11-28T02:04:43.569ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2695 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = ccb33b10-8789-4bda-b261-227a160f1a46 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
50562024-11-28T02:04:44.194ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1341 local_addr = [fd00:1122:3344:101::b]:55621 method = GET remote_addr = [fd00:1122:3344:101::d]:56417 req_id = 6df1f177-80d7-46a6-bec3-0d0a92e84fcd response_code = 200 uri = /2469a748-bcd2-4e0d-99cc-f79def0452cc
50572024-11-28T02:04:44.586ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 477b8ff4-72f1-474d-aa61-87c679a4095e roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
50582024-11-28T02:04:44.587ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 477b8ff4-72f1-474d-aa61-87c679a4095e resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
50592024-11-28T02:04:44.589ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2810 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 477b8ff4-72f1-474d-aa61-87c679a4095e response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
50602024-11-28T02:04:45.607ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 47d9420d-e477-4e68-a920-906dd0042570 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
50612024-11-28T02:04:45.607ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 47d9420d-e477-4e68-a920-906dd0042570 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
50622024-11-28T02:04:45.609ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2656 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 47d9420d-e477-4e68-a920-906dd0042570 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
50632024-11-28T02:04:46.627ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 0ad51837-af63-43f7-b62e-7a566c47d5cc roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
50642024-11-28T02:04:46.627ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 0ad51837-af63-43f7-b62e-7a566c47d5cc resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
50652024-11-28T02:04:46.629ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2242 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 0ad51837-af63-43f7-b62e-7a566c47d5cc response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
50662024-11-28T02:04:47.648ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = f5c62f80-b3ca-4ccc-9902-0d6210501618 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
50672024-11-28T02:04:47.648ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = f5c62f80-b3ca-4ccc-9902-0d6210501618 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
50682024-11-28T02:04:47.650ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2324 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = f5c62f80-b3ca-4ccc-9902-0d6210501618 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
50692024-11-28T02:04:48.669ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = ef9482ac-dbe0-4a3a-bbb0-a0906eeda7f4 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
50702024-11-28T02:04:48.670ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = ef9482ac-dbe0-4a3a-bbb0-a0906eeda7f4 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
50712024-11-28T02:04:48.672ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2407 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = ef9482ac-dbe0-4a3a-bbb0-a0906eeda7f4 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
50722024-11-28T02:04:49.691ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = fd8c566c-b094-44bc-a351-de8bfec42401 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
50732024-11-28T02:04:49.691ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = fd8c566c-b094-44bc-a351-de8bfec42401 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
50742024-11-28T02:04:49.693ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2529 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = fd8c566c-b094-44bc-a351-de8bfec42401 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
50752024-11-28T02:04:50.478ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 71 reason = Timeout
50762024-11-28T02:04:50.478ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 71 reason = Timeout
50772024-11-28T02:04:50.478ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
50782024-11-28T02:04:50.478ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
50792024-11-28T02:04:50.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
50802024-11-28T02:04:50.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
50812024-11-28T02:04:50.480ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
50822024-11-28T02:04:50.483ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
50832024-11-28T02:04:50.483ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
50842024-11-28T02:04:50.483ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
50852024-11-28T02:04:50.485ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
50862024-11-28T02:04:50.485ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
50872024-11-28T02:04:50.486ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
50882024-11-28T02:04:50.489ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 67ac247f-c934-4c78-857a-fba4fbeb9937, lookup_type: ById(67ac247f-c934-4c78-857a-fba4fbeb9937) } result = Ok(())
50892024-11-28T02:04:50.489ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
50902024-11-28T02:04:50.489ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
50912024-11-28T02:04:50.505ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 67ac247f-c934-4c78-857a-fba4fbeb9937 original_time_created = 2024-11-28 01:53:03.159475 UTC target_id = 67ac247f-c934-4c78-857a-fba4fbeb9937 time_created = 2024-11-28 01:53:03.159475 UTC
50922024-11-28T02:04:50.505ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 26.410708ms iteration = 71 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"67ac247f-c934-4c78-857a-fba4fbeb9937","time_created":"2024-11-28T01:53:03.159475Z"}
50932024-11-28T02:04:50.511ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
50942024-11-28T02:04:50.512ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
50952024-11-28T02:04:50.546ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 68.117863ms iteration = 71 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
50962024-11-28T02:04:50.711ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 5f1cc6c1-5a9c-4cd5-9361-2d46e05460ee roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
50972024-11-28T02:04:50.711ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 5f1cc6c1-5a9c-4cd5-9361-2d46e05460ee resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
50982024-11-28T02:04:50.713ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2232 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 5f1cc6c1-5a9c-4cd5-9361-2d46e05460ee response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
50992024-11-28T02:04:51.731ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 4d115c02-8dda-4abe-9cf3-d247e5e75346 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
51002024-11-28T02:04:51.731ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 4d115c02-8dda-4abe-9cf3-d247e5e75346 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
51012024-11-28T02:04:51.734ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2568 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 4d115c02-8dda-4abe-9cf3-d247e5e75346 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
51022024-11-28T02:04:52.753ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 8977b3a9-8934-464d-8e03-14bc82a9ecf6 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
51032024-11-28T02:04:52.753ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 8977b3a9-8934-464d-8e03-14bc82a9ecf6 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
51042024-11-28T02:04:52.755ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2447 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 8977b3a9-8934-464d-8e03-14bc82a9ecf6 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
51052024-11-28T02:04:53.773ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = e3f380bc-a57f-4547-9150-ef2475103126 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
51062024-11-28T02:04:53.773ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = e3f380bc-a57f-4547-9150-ef2475103126 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
51072024-11-28T02:04:53.775ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2258 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = e3f380bc-a57f-4547-9150-ef2475103126 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
51082024-11-28T02:04:54.194ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1251 local_addr = [fd00:1122:3344:101::b]:55621 method = GET remote_addr = [fd00:1122:3344:101::d]:56417 req_id = 13fdcc46-4a80-4168-8cd0-fd65f7272494 response_code = 200 uri = /2469a748-bcd2-4e0d-99cc-f79def0452cc
51092024-11-28T02:04:54.794ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 62ccbda6-d384-4788-9f0a-1706fd24c662 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
51102024-11-28T02:04:54.794ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 62ccbda6-d384-4788-9f0a-1706fd24c662 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
51112024-11-28T02:04:54.796ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2191 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 62ccbda6-d384-4788-9f0a-1706fd24c662 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
51122024-11-28T02:04:55.816ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = b3b12573-5b6c-4e4c-9f54-9494c316a55e roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
51132024-11-28T02:04:55.816ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = b3b12573-5b6c-4e4c-9f54-9494c316a55e resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
51142024-11-28T02:04:55.818ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2540 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = b3b12573-5b6c-4e4c-9f54-9494c316a55e response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
51152024-11-28T02:04:56.836ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = c06e4407-8cbd-4cdc-8c3e-4ed1218cd7e7 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
51162024-11-28T02:04:56.837ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = c06e4407-8cbd-4cdc-8c3e-4ed1218cd7e7 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
51172024-11-28T02:04:56.839ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2457 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = c06e4407-8cbd-4cdc-8c3e-4ed1218cd7e7 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
51182024-11-28T02:04:57.857ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = b59eab68-01bd-47e3-a59c-ba8a18dce355 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
51192024-11-28T02:04:57.858ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = b59eab68-01bd-47e3-a59c-ba8a18dce355 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
51202024-11-28T02:04:57.860ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2634 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = b59eab68-01bd-47e3-a59c-ba8a18dce355 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
51212024-11-28T02:04:58.878ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 94280323-efe5-42cd-8748-c732133d455a roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
51222024-11-28T02:04:58.879ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 94280323-efe5-42cd-8748-c732133d455a resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
51232024-11-28T02:04:58.880ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2075 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 94280323-efe5-42cd-8748-c732133d455a response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
51242024-11-28T02:04:59.899ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = db3f1e6c-ffda-4ba5-9336-f4ed0d1795b4 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
51252024-11-28T02:04:59.899ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = db3f1e6c-ffda-4ba5-9336-f4ed0d1795b4 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
51262024-11-28T02:04:59.901ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2279 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = db3f1e6c-ffda-4ba5-9336-f4ed0d1795b4 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
51272024-11-28T02:05:00.479ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 72 reason = Timeout
51282024-11-28T02:05:00.479ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 72 reason = Timeout
51292024-11-28T02:05:00.479ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
51302024-11-28T02:05:00.479ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
51312024-11-28T02:05:00.480ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
51322024-11-28T02:05:00.480ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
51332024-11-28T02:05:00.506ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
51342024-11-28T02:05:00.508ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
51352024-11-28T02:05:00.508ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
51362024-11-28T02:05:00.508ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
51372024-11-28T02:05:00.509ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
51382024-11-28T02:05:00.509ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
51392024-11-28T02:05:00.510ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
51402024-11-28T02:05:00.510ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 31.380075ms iteration = 72 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
51412024-11-28T02:05:00.511ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
51422024-11-28T02:05:00.512ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
51432024-11-28T02:05:00.515ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 67ac247f-c934-4c78-857a-fba4fbeb9937, lookup_type: ById(67ac247f-c934-4c78-857a-fba4fbeb9937) } result = Ok(())
51442024-11-28T02:05:00.515ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
51452024-11-28T02:05:00.515ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
51462024-11-28T02:05:00.525ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 67ac247f-c934-4c78-857a-fba4fbeb9937 original_time_created = 2024-11-28 01:53:03.159475 UTC target_id = 67ac247f-c934-4c78-857a-fba4fbeb9937 time_created = 2024-11-28 01:53:03.159475 UTC
51472024-11-28T02:05:00.525ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 46.165762ms iteration = 72 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"67ac247f-c934-4c78-857a-fba4fbeb9937","time_created":"2024-11-28T01:53:03.159475Z"}
51482024-11-28T02:05:00.920ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 841209d3-c0e0-4766-9fb9-0db85bddd41d roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
51492024-11-28T02:05:00.920ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 841209d3-c0e0-4766-9fb9-0db85bddd41d resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
51502024-11-28T02:05:00.922ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2161 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 841209d3-c0e0-4766-9fb9-0db85bddd41d response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
51512024-11-28T02:05:01.941ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = c77be71a-4815-46eb-b828-d701d7e80661 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
51522024-11-28T02:05:01.941ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = c77be71a-4815-46eb-b828-d701d7e80661 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
51532024-11-28T02:05:01.943ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2084 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = c77be71a-4815-46eb-b828-d701d7e80661 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
51542024-11-28T02:05:02.962ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = eaece373-35dd-4b51-8ac8-8e94f022c147 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
51552024-11-28T02:05:02.963ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = eaece373-35dd-4b51-8ac8-8e94f022c147 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
51562024-11-28T02:05:02.965ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2614 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = eaece373-35dd-4b51-8ac8-8e94f022c147 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
51572024-11-28T02:05:03.985ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 7b0c8cf3-eb57-4f96-80aa-40cfd6f6c495 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
51582024-11-28T02:05:03.985ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 7b0c8cf3-eb57-4f96-80aa-40cfd6f6c495 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
51592024-11-28T02:05:03.987ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2397 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 7b0c8cf3-eb57-4f96-80aa-40cfd6f6c495 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
51602024-11-28T02:05:04.195ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1883 local_addr = [fd00:1122:3344:101::b]:55621 method = GET remote_addr = [fd00:1122:3344:101::d]:56417 req_id = 3f6d8b15-5135-4b3d-9aff-ed3853a22bf4 response_code = 200 uri = /2469a748-bcd2-4e0d-99cc-f79def0452cc
51612024-11-28T02:05:05.006ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 352b5376-26bb-42cb-9a3b-870eeb52e992 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
51622024-11-28T02:05:05.006ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 352b5376-26bb-42cb-9a3b-870eeb52e992 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
51632024-11-28T02:05:05.008ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2135 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 352b5376-26bb-42cb-9a3b-870eeb52e992 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
51642024-11-28T02:05:06.027ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 45fdbfc4-3827-45c2-a081-5e1cd6fcec79 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
51652024-11-28T02:05:06.027ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 45fdbfc4-3827-45c2-a081-5e1cd6fcec79 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
51662024-11-28T02:05:06.029ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2646 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 45fdbfc4-3827-45c2-a081-5e1cd6fcec79 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
51672024-11-28T02:05:07.048ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = affc7ae7-f7a5-471b-b26d-dabf3d8cd464 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
51682024-11-28T02:05:07.049ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = affc7ae7-f7a5-471b-b26d-dabf3d8cd464 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
51692024-11-28T02:05:07.050ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2233 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = affc7ae7-f7a5-471b-b26d-dabf3d8cd464 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
51702024-11-28T02:05:08.069ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 6a286b05-f696-4b38-bcf1-98866d105252 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
51712024-11-28T02:05:08.070ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 6a286b05-f696-4b38-bcf1-98866d105252 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
51722024-11-28T02:05:08.072ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2202 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 6a286b05-f696-4b38-bcf1-98866d105252 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
51732024-11-28T02:05:09.090ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 4d3ea3db-16b6-4bec-a0ab-2335f01f37b6 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
51742024-11-28T02:05:09.090ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 4d3ea3db-16b6-4bec-a0ab-2335f01f37b6 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
51752024-11-28T02:05:09.092ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2351 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 4d3ea3db-16b6-4bec-a0ab-2335f01f37b6 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
51762024-11-28T02:05:09.507ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:40402 req_id = 3a006cf3-2fe9-474e-b700-14b0e66a89c9 roles = RoleSet { roles: {} } uri = /nat/ipv4/changeset/10?limit=100
51772024-11-28T02:05:09.507ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:40402 req_id = 3a006cf3-2fe9-474e-b700-14b0e66a89c9 resource = Database result = Ok(()) uri = /nat/ipv4/changeset/10?limit=100
51782024-11-28T02:05:09.510ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2808 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:40402 req_id = 3a006cf3-2fe9-474e-b700-14b0e66a89c9 response_code = 200 uri = /nat/ipv4/changeset/10?limit=100
51792024-11-28T02:05:10.112ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = b7a4c6f6-3c74-414e-afac-a1c806deee74 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
51802024-11-28T02:05:10.112ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = b7a4c6f6-3c74-414e-afac-a1c806deee74 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
51812024-11-28T02:05:10.114ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2188 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = b7a4c6f6-3c74-414e-afac-a1c806deee74 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
51822024-11-28T02:05:10.478ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 15 reason = Timeout
51832024-11-28T02:05:10.478ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 25 reason = Timeout
51842024-11-28T02:05:10.478ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 13 reason = Timeout
51852024-11-28T02:05:10.478ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
51862024-11-28T02:05:10.478ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
51872024-11-28T02:05:10.479ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 25 reason = Timeout
51882024-11-28T02:05:10.479ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
51892024-11-28T02:05:10.479ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 14 reason = Timeout
51902024-11-28T02:05:10.479ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
51912024-11-28T02:05:10.479ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
51922024-11-28T02:05:10.479ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 26 reason = Timeout
51932024-11-28T02:05:10.479ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
51942024-11-28T02:05:10.479ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 13 reason = Timeout
51952024-11-28T02:05:10.479ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2024-11-28 01:55:10.479020630 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
51962024-11-28T02:05:10.479ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 73 reason = Timeout
51972024-11-28T02:05:10.479ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
51982024-11-28T02:05:10.479ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
51992024-11-28T02:05:10.479ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 14 reason = Timeout
52002024-11-28T02:05:10.479ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 25 reason = Timeout
52012024-11-28T02:05:10.479ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
52022024-11-28T02:05:10.479ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 14 reason = Timeout
52032024-11-28T02:05:10.479ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 25 reason = Timeout
52042024-11-28T02:05:10.479ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 73 reason = Timeout
52052024-11-28T02:05:10.479ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
52062024-11-28T02:05:10.479ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 13 reason = Timeout
52072024-11-28T02:05:10.479ZDEBGnexus (BackgroundTasks): activating background_task = crdb_node_id_collector iteration = 6 reason = Timeout
52082024-11-28T02:05:10.479ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
52092024-11-28T02:05:10.479ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
52102024-11-28T02:05:10.479ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 27 reason = Timeout
52112024-11-28T02:05:10.479ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 25 reason = Timeout
52122024-11-28T02:05:10.479ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
52132024-11-28T02:05:10.479ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
52142024-11-28T02:05:10.479ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
52152024-11-28T02:05:10.479ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 25 reason = Timeout
52162024-11-28T02:05:10.479ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
52172024-11-28T02:05:10.479ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 25 reason = Timeout
52182024-11-28T02:05:10.479ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
52192024-11-28T02:05:10.479ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 15 reason = Timeout
52202024-11-28T02:05:10.479ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 14 reason = Timeout
52212024-11-28T02:05:10.479ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
52222024-11-28T02:05:10.479ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 25 reason = Timeout
52232024-11-28T02:05:10.479ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
52242024-11-28T02:05:10.479ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
52252024-11-28T02:05:10.479ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 13 reason = Timeout
52262024-11-28T02:05:10.479ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:56
52272024-11-28T02:05:10.479ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 29 reason = Timeout
52282024-11-28T02:05:10.479ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
52292024-11-28T02:05:10.479ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 14 reason = Timeout
52302024-11-28T02:05:10.479ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
52312024-11-28T02:05:10.479ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
52322024-11-28T02:05:10.479ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 14 reason = Timeout
52332024-11-28T02:05:10.479ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:83 reason = blueprint disabled target_id = 67ac247f-c934-4c78-857a-fba4fbeb9937
52342024-11-28T02:05:10.479ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 13.841\u{b5}s iteration = 14 status = {"enabled":false,"target_id":"67ac247f-c934-4c78-857a-fba4fbeb9937"}
52352024-11-28T02:05:10.479ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 25 reason = Timeout
52362024-11-28T02:05:10.479ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
52372024-11-28T02:05:10.479ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 25 reason = Timeout
52382024-11-28T02:05:10.479ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
52392024-11-28T02:05:10.479ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
52402024-11-28T02:05:10.479ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 13 reason = Timeout
52412024-11-28T02:05:10.479ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
52422024-11-28T02:05:10.479ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 25 reason = Timeout
52432024-11-28T02:05:10.479ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 25 reason = Timeout
52442024-11-28T02:05:10.479ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
52452024-11-28T02:05:10.479ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
52462024-11-28T02:05:10.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
52472024-11-28T02:05:10.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
52482024-11-28T02:05:10.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
52492024-11-28T02:05:10.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
52502024-11-28T02:05:10.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
52512024-11-28T02:05:10.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
52522024-11-28T02:05:10.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
52532024-11-28T02:05:10.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
52542024-11-28T02:05:10.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
52552024-11-28T02:05:10.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
52562024-11-28T02:05:10.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
52572024-11-28T02:05:10.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
52582024-11-28T02:05:10.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
52592024-11-28T02:05:10.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
52602024-11-28T02:05:10.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
52612024-11-28T02:05:10.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
52622024-11-28T02:05:10.479ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
52632024-11-28T02:05:10.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
52642024-11-28T02:05:10.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
52652024-11-28T02:05:10.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
52662024-11-28T02:05:10.480ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
52672024-11-28T02:05:10.480ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
52682024-11-28T02:05:10.480ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
52692024-11-28T02:05:10.480ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
52702024-11-28T02:05:10.480ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
52712024-11-28T02:05:10.480ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
52722024-11-28T02:05:10.480ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("36d4f615-7461-4364-960c-89ece3aaabaf.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("eb274350-c0b0-4bce-9c8a-502880f2c905.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("eb274350-c0b0-4bce-9c8a-502880f2c905.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::8))) }], valid_until: Instant { tv_sec: 1576, tv_nsec: 950038076 } })
52732024-11-28T02:05:10.480ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("41ea4a99-3038-474e-bf28-2ad338f3243f.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("9402a51b-d30f-4544-93ec-0f5dc7a485e4.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("d23bc5f2-6b92-4992-bdd2-8695e5bf9c0f.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("d23bc5f2-6b92-4992-bdd2-8695e5bf9c0f.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:1::1))) }], valid_until: Instant { tv_sec: 1576, tv_nsec: 950062227 } })
52742024-11-28T02:05:10.480ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
52752024-11-28T02:05:10.480ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
52762024-11-28T02:05:10.480ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
52772024-11-28T02:05:10.480ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
52782024-11-28T02:05:10.480ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
52792024-11-28T02:05:10.480ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
52802024-11-28T02:05:10.480ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } dns_group = external
52812024-11-28T02:05:10.480ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.525397ms iteration = 14 status = {"addresses":["[fd00:1122:3344:101::9]:5353","[fd00:1122:3344:101::8]:5353"]}
52822024-11-28T02:05:10.480ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } dns_group = internal
52832024-11-28T02:05:10.480ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.572787ms iteration = 14 status = {"addresses":["[fd00:1122:3344:3::1]:5353","[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:1::1]:5353"]}
52842024-11-28T02:05:10.480ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
52852024-11-28T02:05:10.482ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2024-11-28 01:55:10.479020630 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
52862024-11-28T02:05:10.482ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 3.280457ms iteration = 13 status = {"expiration":"2024-11-28T01:55:10.479020630Z","pruned":[]}
52872024-11-28T02:05:10.483ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.97842ms iteration = 25 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
52882024-11-28T02:05:10.483ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
52892024-11-28T02:05:10.483ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
52902024-11-28T02:05:10.484ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-313847b3-d670-4167-9b09-9e7d3dfcb2a4.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-313847b3-d670-4167-9b09-9e7d3dfcb2a4.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1576, tv_nsec: 954834580 } })
52912024-11-28T02:05:10.485ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1103
52922024-11-28T02:05:10.487ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
52932024-11-28T02:05:10.488ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
52942024-11-28T02:05:10.490ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
52952024-11-28T02:05:10.490ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
52962024-11-28T02:05:10.491ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 12.399465ms iteration = 13 status = {"errors":[],"found_port_ok":[]}
52972024-11-28T02:05:10.492ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-313847b3-d670-4167-9b09-9e7d3dfcb2a4.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-313847b3-d670-4167-9b09-9e7d3dfcb2a4.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1576, tv_nsec: 962947511 } })
52982024-11-28T02:05:10.493ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1103
52992024-11-28T02:05:10.538ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
53002024-11-28T02:05:10.542ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
53012024-11-28T02:05:10.542ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 63.86043ms iteration = 25 status = {"errors":[],"garbage_collect_requested":[]}
53022024-11-28T02:05:10.542ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 63.783838ms iteration = 25 status = {"errors":[],"records_set_to_done":[]}
53032024-11-28T02:05:10.543ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
53042024-11-28T02:05:10.543ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
53052024-11-28T02:05:10.543ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
53062024-11-28T02:05:10.543ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
53072024-11-28T02:05:10.543ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
53082024-11-28T02:05:10.543ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:171 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
53092024-11-28T02:05:10.543ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 64.551081ms iteration = 13 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
53102024-11-28T02:05:10.544ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
53112024-11-28T02:05:10.544ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
53122024-11-28T02:05:10.545ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
53132024-11-28T02:05:10.545ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
53142024-11-28T02:05:10.545ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
53152024-11-28T02:05:10.545ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
53162024-11-28T02:05:10.546ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
53172024-11-28T02:05:10.546ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
53182024-11-28T02:05:10.546ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
53192024-11-28T02:05:10.546ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
53202024-11-28T02:05:10.546ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
53212024-11-28T02:05:10.548ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
53222024-11-28T02:05:10.549ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
53232024-11-28T02:05:10.549ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
53242024-11-28T02:05:10.549ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
53252024-11-28T02:05:10.549ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
53262024-11-28T02:05:10.549ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
53272024-11-28T02:05:10.549ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
53282024-11-28T02:05:10.550ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
53292024-11-28T02:05:10.550ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
53302024-11-28T02:05:10.550ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
53312024-11-28T02:05:10.551ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
53322024-11-28T02:05:10.551ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
53332024-11-28T02:05:10.552ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
53342024-11-28T02:05:10.552ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
53352024-11-28T02:05:10.552ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
53362024-11-28T02:05:10.552ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
53372024-11-28T02:05:10.553ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
53382024-11-28T02:05:10.553ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
53392024-11-28T02:05:10.553ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
53402024-11-28T02:05:10.553ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
53412024-11-28T02:05:10.554ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
53422024-11-28T02:05:10.554ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
53432024-11-28T02:05:10.555ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
53442024-11-28T02:05:10.555ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
53452024-11-28T02:05:10.555ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
53462024-11-28T02:05:10.555ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
53472024-11-28T02:05:10.556ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
53482024-11-28T02:05:10.556ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
53492024-11-28T02:05:10.556ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
53502024-11-28T02:05:10.557ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
53512024-11-28T02:05:10.557ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
53522024-11-28T02:05:10.557ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
53532024-11-28T02:05:10.557ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
53542024-11-28T02:05:10.557ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
53552024-11-28T02:05:10.558ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
53562024-11-28T02:05:10.558ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
53572024-11-28T02:05:10.560ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
53582024-11-28T02:05:10.560ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
53592024-11-28T02:05:10.560ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
53602024-11-28T02:05:10.560ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
53612024-11-28T02:05:10.561ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
53622024-11-28T02:05:10.563ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
53632024-11-28T02:05:10.564ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
53642024-11-28T02:05:10.564ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
53652024-11-28T02:05:10.565ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
53662024-11-28T02:05:10.566ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
53672024-11-28T02:05:10.566ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
53682024-11-28T02:05:10.566ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
53692024-11-28T02:05:10.567ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
53702024-11-28T02:05:10.567ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
53712024-11-28T02:05:10.569ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
53722024-11-28T02:05:10.569ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
53732024-11-28T02:05:10.569ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
53742024-11-28T02:05:10.571ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
53752024-11-28T02:05:10.572ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
53762024-11-28T02:05:10.572ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-313847b3-d670-4167-9b09-9e7d3dfcb2a4.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-313847b3-d670-4167-9b09-9e7d3dfcb2a4.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1577, tv_nsec: 42290189 } })
53772024-11-28T02:05:10.572ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1103 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc
53782024-11-28T02:05:10.573ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
53792024-11-28T02:05:10.573ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
53802024-11-28T02:05:10.576ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
53812024-11-28T02:05:10.576ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
53822024-11-28T02:05:10.576ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
53832024-11-28T02:05:10.576ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
53842024-11-28T02:05:10.576ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
53852024-11-28T02:05:10.576ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
53862024-11-28T02:05:10.576ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
53872024-11-28T02:05:10.578ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
53882024-11-28T02:05:10.578ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
53892024-11-28T02:05:10.578ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
53902024-11-28T02:05:10.578ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
53912024-11-28T02:05:10.582ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 67ac247f-c934-4c78-857a-fba4fbeb9937, lookup_type: ById(67ac247f-c934-4c78-857a-fba4fbeb9937) } result = Ok(())
53922024-11-28T02:05:10.582ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
53932024-11-28T02:05:10.582ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
53942024-11-28T02:05:10.585ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
53952024-11-28T02:05:10.585ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
53962024-11-28T02:05:10.599ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
53972024-11-28T02:05:10.603ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
53982024-11-28T02:05:10.603ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
53992024-11-28T02:05:10.603ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
54002024-11-28T02:05:10.603ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
54012024-11-28T02:05:10.606ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
54022024-11-28T02:05:10.606ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
54032024-11-28T02:05:10.607ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
54042024-11-28T02:05:10.609ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
54052024-11-28T02:05:10.609ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
54062024-11-28T02:05:10.610ZDEBGnexus (BackgroundTasks): activation complete background_task = crdb_node_id_collector elapsed = 130.989485ms iteration = 6 status = {"nsuccess":5}
54072024-11-28T02:05:10.611ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 67ac247f-c934-4c78-857a-fba4fbeb9937 original_time_created = 2024-11-28 01:53:03.159475 UTC target_id = 67ac247f-c934-4c78-857a-fba4fbeb9937 time_created = 2024-11-28 01:53:03.159475 UTC
54082024-11-28T02:05:10.611ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 132.353019ms iteration = 73 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"67ac247f-c934-4c78-857a-fba4fbeb9937","time_created":"2024-11-28T01:53:03.159475Z"}
54092024-11-28T02:05:10.611ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
54102024-11-28T02:05:10.611ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
54112024-11-28T02:05:10.611ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
54122024-11-28T02:05:10.611ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
54132024-11-28T02:05:10.612ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
54142024-11-28T02:05:10.613ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
54152024-11-28T02:05:10.614ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
54162024-11-28T02:05:10.614ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
54172024-11-28T02:05:10.614ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
54182024-11-28T02:05:10.614ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
54192024-11-28T02:05:10.614ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
54202024-11-28T02:05:10.615ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 136.237647ms iteration = 25 status = {"errors":[],"requests_created_ok":[],"start_invoked_ok":[]}
54212024-11-28T02:05:10.616ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
54222024-11-28T02:05:10.616ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 137.282435ms iteration = 25 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
54232024-11-28T02:05:10.616ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
54242024-11-28T02:05:10.616ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
54252024-11-28T02:05:10.616ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
54262024-11-28T02:05:10.616ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
54272024-11-28T02:05:10.616ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
54282024-11-28T02:05:10.617ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 138.415334ms iteration = 73 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
54292024-11-28T02:05:10.619ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
54302024-11-28T02:05:10.619ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
54312024-11-28T02:05:10.619ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
54322024-11-28T02:05:10.622ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
54332024-11-28T02:05:10.622ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
54342024-11-28T02:05:10.627ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
54352024-11-28T02:05:10.627ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:140 found = 0 not_ready_to_be_deleted = 0
54362024-11-28T02:05:10.627ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 147.832037ms iteration = 13 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
54372024-11-28T02:05:10.628ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
54382024-11-28T02:05:10.628ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 148.899307ms iteration = 25 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
54392024-11-28T02:05:10.633ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
54402024-11-28T02:05:10.633ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
54412024-11-28T02:05:10.633ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 154.003045ms iteration = 25 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
54422024-11-28T02:05:10.637ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
54432024-11-28T02:05:10.638ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
54442024-11-28T02:05:10.639ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
54452024-11-28T02:05:10.640ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
54462024-11-28T02:05:10.640ZWARNnexus (BackgroundTasks): instance reincarnation completed with errors background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:103 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0 query_errors = 0 restart_errors = 0
54472024-11-28T02:05:10.640ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 161.694158ms iteration = 13 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
54482024-11-28T02:05:10.641ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
54492024-11-28T02:05:10.641ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
54502024-11-28T02:05:10.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
54512024-11-28T02:05:10.653ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
54522024-11-28T02:05:10.653ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 808a324e-c913-4bfc-91a5-d43c961cd511 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
54532024-11-28T02:05:10.653ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
54542024-11-28T02:05:10.653ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
54552024-11-28T02:05:10.654ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
54562024-11-28T02:05:10.656ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 177.751118ms iteration = 25 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[]}
54572024-11-28T02:05:10.657ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
54582024-11-28T02:05:10.657ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = a7e6af61-408c-483d-b043-c27ccf55708e dns_zone_name = oxide.test found_so_far = 0 version = 2
54592024-11-28T02:05:10.657ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
54602024-11-28T02:05:10.657ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
54612024-11-28T02:05:10.659ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
54622024-11-28T02:05:10.659ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
54632024-11-28T02:05:10.659ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
54642024-11-28T02:05:10.661ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
54652024-11-28T02:05:10.662ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 808a324e-c913-4bfc-91a5-d43c961cd511 dns_zone_name = control-plane.oxide.internal found_so_far = 51 version = 1
54662024-11-28T02:05:10.662ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
54672024-11-28T02:05:10.662ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2024-11-28 01:53:42.502562 UTC dns_group = internal generation = 1
54682024-11-28T02:05:10.662ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 183.316664ms iteration = 14 status = {"generation":1}
54692024-11-28T02:05:10.663ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
54702024-11-28T02:05:10.663ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
54712024-11-28T02:05:10.664ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
54722024-11-28T02:05:10.664ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
54732024-11-28T02:05:10.666ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = a7e6af61-408c-483d-b043-c27ccf55708e dns_zone_name = oxide.test found_so_far = 1 version = 2
54742024-11-28T02:05:10.666ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
54752024-11-28T02:05:10.666ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2024-11-28 01:53:43.351579 UTC dns_group = external generation = 2
54762024-11-28T02:05:10.666ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 187.020839ms iteration = 14 status = {"generation":2}
54772024-11-28T02:05:10.667ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 313847b3-d670-4167-9b09-9e7d3dfcb2a4, lookup_type: ById(313847b3-d670-4167-9b09-9e7d3dfcb2a4) } result = Ok(())
54782024-11-28T02:05:10.667ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
54792024-11-28T02:05:10.667ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
54802024-11-28T02:05:10.669ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 190.113152ms iteration = 25 status = 0
54812024-11-28T02:05:10.683ZINFOnexus (BackgroundTasks): found Silo/DNS/TLS config (changed) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2024-11-28T01:53:08.890632Z, time_modified: 2024-11-28T01:53:08.890632Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 7e159834-997a-4c03-b9db-1e91d783221c, db_silo: Silo { identity: SiloIdentity { id: 7e159834-997a-4c03-b9db-1e91d783221c, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-11-28T01:53:43.308696Z, time_modified: 2024-11-28T01:53:43.308696Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "bc39d1b3b4bb84a49d6458ed7aedc5bca9e77742c7b35882b64d188c0602d306" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 7e159834-997a-4c03-b9db-1e91d783221c, db_silo: Silo { identity: SiloIdentity { id: 7e159834-997a-4c03-b9db-1e91d783221c, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-11-28T01:53:43.308696Z, time_modified: 2024-11-28T01:53:43.308696Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "bc39d1b3b4bb84a49d6458ed7aedc5bca9e77742c7b35882b64d188c0602d306" }] }) } file = nexus/src/app/background/tasks/external_endpoints.rs:94
54822024-11-28T02:05:10.683ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 203.83261ms iteration = 14 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"7e159834-997a-4c03-b9db-1e91d783221c","tls_certs":["bc39d1b3b4bb84a49d6458ed7aedc5bca9e77742c7b35882b64d188c0602d306"]}},"default_endpoint":{"silo_id":"7e159834-997a-4c03-b9db-1e91d783221c","tls_certs":["bc39d1b3b4bb84a49d6458ed7aedc5bca9e77742c7b35882b64d188c0602d306"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
54832024-11-28T02:05:10.707ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
54842024-11-28T02:05:10.707ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
54852024-11-28T02:05:10.710ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
54862024-11-28T02:05:10.713ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
54872024-11-28T02:05:10.713ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
54882024-11-28T02:05:10.714ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
54892024-11-28T02:05:10.715ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
54902024-11-28T02:05:10.719ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
54912024-11-28T02:05:10.719ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
54922024-11-28T02:05:10.719ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
54932024-11-28T02:05:10.732ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 253.273169ms iteration = 29 status = {"physical_disks_added":0}
54942024-11-28T02:05:10.740ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
54952024-11-28T02:05:10.740ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
54962024-11-28T02:05:10.740ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "f29a867e-e78f-4bf5-9bed-3fcd6f5f4ac2", "content-length": "26", "date": "Thu, 28 Nov 2024 02:05:10 GMT"} })
54972024-11-28T02:05:10.740ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
54982024-11-28T02:05:10.742ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
54992024-11-28T02:05:10.766ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
55002024-11-28T02:05:10.766ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
55012024-11-28T02:05:10.767ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "57036555-66a3-4869-b3f4-0974e6401d3c", "content-length": "26", "date": "Thu, 28 Nov 2024 02:05:10 GMT"} })
55022024-11-28T02:05:10.767ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
55032024-11-28T02:05:10.769ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
55042024-11-28T02:05:10.783ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
55052024-11-28T02:05:10.812ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1111 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc zone_address = fd00:1122:3344:101::2
55062024-11-28T02:05:10.812ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
55072024-11-28T02:05:10.812ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "bc684cc7-ad96-40c2-bbe1-368a85b90f7b", "content-length": "26", "date": "Thu, 28 Nov 2024 02:05:10 GMT"} })
55082024-11-28T02:05:10.812ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1116 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
55092024-11-28T02:05:10.814ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc
55102024-11-28T02:05:10.845ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
55112024-11-28T02:05:10.846ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "ffc12a7a-f265-4760-acab-6b398ada3bd9", "date": "Thu, 28 Nov 2024 02:05:10 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
55122024-11-28T02:05:10.846ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
55132024-11-28T02:05:10.847ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "e34b67bf-8299-433f-832f-a77843c62049", "date": "Thu, 28 Nov 2024 02:05:10 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
55142024-11-28T02:05:10.847ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
55152024-11-28T02:05:10.849ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
55162024-11-28T02:05:10.849ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 370.76865ms iteration = 15 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
55172024-11-28T02:05:10.876ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
55182024-11-28T02:05:10.877ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "128aba20-4c99-4c28-8ff7-7a652144c560", "content-length": "2", "date": "Thu, 28 Nov 2024 02:05:10 GMT"} })
55192024-11-28T02:05:10.878ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
55202024-11-28T02:05:10.879ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
55212024-11-28T02:05:10.931ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 452.213695ms iteration = 25 status = {}
55222024-11-28T02:05:10.955ZINFOnexus (BackgroundTasks): VPC route manager sled 313847b3-d670-4167-9b09-9e7d3dfcb2a4 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
55232024-11-28T02:05:10.956ZDEBGnexus (BackgroundTasks): client request SledAgent = 313847b3-d670-4167-9b09-9e7d3dfcb2a4 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
55242024-11-28T02:05:10.956ZDEBGnexus (BackgroundTasks): client response SledAgent = 313847b3-d670-4167-9b09-9e7d3dfcb2a4 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "6594eeaf-3788-4d69-be37-252c3a4912d3", "content-length": "372", "date": "Thu, 28 Nov 2024 02:05:10 GMT"} })
55252024-11-28T02:05:10.956ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
55262024-11-28T02:05:10.956ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
55272024-11-28T02:05:10.959ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 479.754943ms iteration = 25 status = null
55282024-11-28T02:05:10.987ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
55292024-11-28T02:05:10.999ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
55302024-11-28T02:05:10.999ZDEBGnexus (BackgroundTasks): client request SledAgent = 313847b3-d670-4167-9b09-9e7d3dfcb2a4 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
55312024-11-28T02:05:10.999ZDEBGnexus (BackgroundTasks): client response SledAgent = 313847b3-d670-4167-9b09-9e7d3dfcb2a4 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "e70e05ef-8f77-4c48-aa2f-f6e60ae8483f", "date": "Thu, 28 Nov 2024 02:05:10 GMT"} })
55322024-11-28T02:05:10.999ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
55332024-11-28T02:05:10.999ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
55342024-11-28T02:05:11.001ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
55352024-11-28T02:05:11.001ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
55362024-11-28T02:05:11.004ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
55372024-11-28T02:05:11.004ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
55382024-11-28T02:05:11.008ZINFOnexus (BackgroundTasks): VPC route manager sled 313847b3-d670-4167-9b09-9e7d3dfcb2a4 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
55392024-11-28T02:05:11.008ZDEBGnexus (BackgroundTasks): client request SledAgent = 313847b3-d670-4167-9b09-9e7d3dfcb2a4 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
55402024-11-28T02:05:11.097ZDEBGnexus (BackgroundTasks): client response SledAgent = 313847b3-d670-4167-9b09-9e7d3dfcb2a4 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "7489429b-f3aa-461c-94ac-b82497a57f7f", "date": "Thu, 28 Nov 2024 02:05:11 GMT"} })
55412024-11-28T02:05:11.100ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 621.339153ms iteration = 27 status = {}
55422024-11-28T02:05:11.101ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
55432024-11-28T02:05:11.102ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
55442024-11-28T02:05:11.104ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 4bb2986f-2e92-4ac3-9dc1-591b6bb60b3e, rack_id: 92a8ff6e-21d3-4c28-bcad-e323847985dc, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(ea300318-9d62-42ed-80ea-21c86590a166) } rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc switch_location = Switch0
55452024-11-28T02:05:11.104ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
55462024-11-28T02:05:11.104ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
55472024-11-28T02:05:11.115ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
55482024-11-28T02:05:11.115ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "a1412cc9-1136-49ae-83de-00731ea07cd0", "content-length": "134", "date": "Thu, 28 Nov 2024 02:05:10 GMT"} })
55492024-11-28T02:05:11.115ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1746 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
55502024-11-28T02:05:11.115ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1773 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = ea300318-9d62-42ed-80ea-21c86590a166
55512024-11-28T02:05:11.116ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
55522024-11-28T02:05:11.116ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
55532024-11-28T02:05:11.116ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
55542024-11-28T02:05:11.116ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
55552024-11-28T02:05:11.116ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
55562024-11-28T02:05:11.116ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1920 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
55572024-11-28T02:05:11.116ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
55582024-11-28T02:05:11.116ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
55592024-11-28T02:05:11.116ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc
55602024-11-28T02:05:11.116ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
55612024-11-28T02:05:11.116ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
55622024-11-28T02:05:11.118ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
55632024-11-28T02:05:11.118ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "7f50b672-6b4a-4fc8-a619-df0476eaeca2", "content-length": "2", "date": "Thu, 28 Nov 2024 02:05:10 GMT"} })
55642024-11-28T02:05:11.118ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
55652024-11-28T02:05:11.118ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "90aa687a-6c3c-4129-a3eb-47b10af86b39", "content-length": "52", "date": "Thu, 28 Nov 2024 02:05:10 GMT"} })
55662024-11-28T02:05:11.118ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc switch_location = Switch0
55672024-11-28T02:05:11.118ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
55682024-11-28T02:05:11.129ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
55692024-11-28T02:05:11.129ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "d60e85d3-c738-4612-bfcc-24937f5ba49b", "date": "Thu, 28 Nov 2024 02:05:10 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
55702024-11-28T02:05:11.129ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
55712024-11-28T02:05:11.131ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "5d089923-985e-410a-96c8-faaee765f75e", "date": "Thu, 28 Nov 2024 02:05:10 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
55722024-11-28T02:05:11.131ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
55732024-11-28T02:05:11.132ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "a115fdea-679a-4cd9-84a5-b564c74299a1", "date": "Thu, 28 Nov 2024 02:05:10 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
55742024-11-28T02:05:11.132ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
55752024-11-28T02:05:11.133ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
55762024-11-28T02:05:11.134ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 654.764603ms iteration = 15 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
55772024-11-28T02:05:11.137ZDEBGnexus (BackgroundTasks): client request SledAgent = 313847b3-d670-4167-9b09-9e7d3dfcb2a4 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
55782024-11-28T02:05:11.214ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "7075a17a-6187-4503-8aa7-7e8eba46b213", "date": "Thu, 28 Nov 2024 02:05:10 GMT"} })
55792024-11-28T02:05:11.214ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc switch = Switch0
55802024-11-28T02:05:11.214ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
55812024-11-28T02:05:11.215ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 3ed5d162-ac7b-445c-8b20-910dff9ee0d0 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
55822024-11-28T02:05:11.215ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 3ed5d162-ac7b-445c-8b20-910dff9ee0d0 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
55832024-11-28T02:05:11.215ZDEBGnexus (BackgroundTasks): client response SledAgent = 313847b3-d670-4167-9b09-9e7d3dfcb2a4 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "863d6767-76cb-4c60-b40b-0a1be95c124c", "content-length": "691", "date": "Thu, 28 Nov 2024 02:05:11 GMT"} })
55842024-11-28T02:05:11.215ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "8746a2fa-d557-4dd9-9bab-049ec8119df7", "content-length": "682", "date": "Thu, 28 Nov 2024 02:05:11 GMT"} })
55852024-11-28T02:05:11.215ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
55862024-11-28T02:05:11.215ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
55872024-11-28T02:05:11.215ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
55882024-11-28T02:05:11.215ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
55892024-11-28T02:05:11.216ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
55902024-11-28T02:05:11.217ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 738.542808ms iteration = 26 status = {}
55912024-11-28T02:05:11.217ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2561 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 3ed5d162-ac7b-445c-8b20-910dff9ee0d0 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
55922024-11-28T02:05:11.218ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
55932024-11-28T02:05:11.218ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
55942024-11-28T02:05:11.221ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
55952024-11-28T02:05:11.221ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
55962024-11-28T02:05:11.223ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
55972024-11-28T02:05:11.223ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
55982024-11-28T02:05:11.230ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 751.408801ms iteration = 25 status = {}
55992024-11-28T02:05:12.322ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = ce43ceee-7264-4dc3-83d8-f38a2ff3f732 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
56002024-11-28T02:05:12.323ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = ce43ceee-7264-4dc3-83d8-f38a2ff3f732 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
56012024-11-28T02:05:12.327ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 4937 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = ce43ceee-7264-4dc3-83d8-f38a2ff3f732 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
56022024-11-28T02:05:13.347ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = c4497668-4702-49a1-8eb6-998ad9fbf6c8 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
56032024-11-28T02:05:13.347ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = c4497668-4702-49a1-8eb6-998ad9fbf6c8 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
56042024-11-28T02:05:13.349ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2498 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = c4497668-4702-49a1-8eb6-998ad9fbf6c8 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
56052024-11-28T02:05:14.194ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1354 local_addr = [fd00:1122:3344:101::b]:55621 method = GET remote_addr = [fd00:1122:3344:101::d]:56417 req_id = 296912d8-3be1-4f04-b63b-7502b88cf30d response_code = 200 uri = /2469a748-bcd2-4e0d-99cc-f79def0452cc
56062024-11-28T02:05:14.368ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 68a4bb44-05ba-40bf-8551-acc844c75ca6 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
56072024-11-28T02:05:14.369ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 68a4bb44-05ba-40bf-8551-acc844c75ca6 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
56082024-11-28T02:05:14.370ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2160 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 68a4bb44-05ba-40bf-8551-acc844c75ca6 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
56092024-11-28T02:05:15.390ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 254e623e-f7a4-4fe7-b922-63a2bd552a9d roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
56102024-11-28T02:05:15.390ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 254e623e-f7a4-4fe7-b922-63a2bd552a9d resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
56112024-11-28T02:05:15.392ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2148 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 254e623e-f7a4-4fe7-b922-63a2bd552a9d response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
56122024-11-28T02:05:16.410ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 913c4014-e4a2-47f6-b4c1-d476d9e1ff67 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
56132024-11-28T02:05:16.410ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 913c4014-e4a2-47f6-b4c1-d476d9e1ff67 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
56142024-11-28T02:05:16.413ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 3119 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 913c4014-e4a2-47f6-b4c1-d476d9e1ff67 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
56152024-11-28T02:05:17.432ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 64548b44-11a0-4b47-93e6-3ec4eef64946 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
56162024-11-28T02:05:17.433ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 64548b44-11a0-4b47-93e6-3ec4eef64946 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
56172024-11-28T02:05:17.435ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2458 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 64548b44-11a0-4b47-93e6-3ec4eef64946 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
56182024-11-28T02:05:18.454ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 93a09ea0-4d71-469a-9fe3-55201f35376e roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
56192024-11-28T02:05:18.454ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 93a09ea0-4d71-469a-9fe3-55201f35376e resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
56202024-11-28T02:05:18.456ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2333 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 93a09ea0-4d71-469a-9fe3-55201f35376e response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
56212024-11-28T02:05:19.476ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = b69219b6-46d2-4430-907a-1a054fb28b33 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
56222024-11-28T02:05:19.476ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = b69219b6-46d2-4430-907a-1a054fb28b33 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
56232024-11-28T02:05:19.477ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1946 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = b69219b6-46d2-4430-907a-1a054fb28b33 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
56242024-11-28T02:05:20.478ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 74 reason = Timeout
56252024-11-28T02:05:20.479ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 74 reason = Timeout
56262024-11-28T02:05:20.479ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
56272024-11-28T02:05:20.479ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
56282024-11-28T02:05:20.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
56292024-11-28T02:05:20.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
56302024-11-28T02:05:20.485ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
56312024-11-28T02:05:20.487ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
56322024-11-28T02:05:20.487ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
56332024-11-28T02:05:20.487ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
56342024-11-28T02:05:20.489ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
56352024-11-28T02:05:20.489ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
56362024-11-28T02:05:20.491ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
56372024-11-28T02:05:20.493ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 67ac247f-c934-4c78-857a-fba4fbeb9937, lookup_type: ById(67ac247f-c934-4c78-857a-fba4fbeb9937) } result = Ok(())
56382024-11-28T02:05:20.493ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
56392024-11-28T02:05:20.493ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
56402024-11-28T02:05:20.496ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 0565888b-d3ab-4bcc-a10c-7eb056a1243f roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
56412024-11-28T02:05:20.496ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 0565888b-d3ab-4bcc-a10c-7eb056a1243f resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
56422024-11-28T02:05:20.498ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1985 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 0565888b-d3ab-4bcc-a10c-7eb056a1243f response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
56432024-11-28T02:05:20.507ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 67ac247f-c934-4c78-857a-fba4fbeb9937 original_time_created = 2024-11-28 01:53:03.159475 UTC target_id = 67ac247f-c934-4c78-857a-fba4fbeb9937 time_created = 2024-11-28 01:53:03.159475 UTC
56442024-11-28T02:05:20.507ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 28.274172ms iteration = 74 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"67ac247f-c934-4c78-857a-fba4fbeb9937","time_created":"2024-11-28T01:53:03.159475Z"}
56452024-11-28T02:05:20.510ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
56462024-11-28T02:05:20.510ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
56472024-11-28T02:05:20.512ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 33.057634ms iteration = 74 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
56482024-11-28T02:05:21.517ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 2ff167f5-6c4a-4a6c-a0cc-56951883a2bc roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
56492024-11-28T02:05:21.518ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 2ff167f5-6c4a-4a6c-a0cc-56951883a2bc resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
56502024-11-28T02:05:21.520ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2536 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 2ff167f5-6c4a-4a6c-a0cc-56951883a2bc response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
56512024-11-28T02:05:22.539ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = fd3e1294-9f34-41a6-8e32-9506a0721f60 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
56522024-11-28T02:05:22.539ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = fd3e1294-9f34-41a6-8e32-9506a0721f60 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
56532024-11-28T02:05:22.541ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2275 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = fd3e1294-9f34-41a6-8e32-9506a0721f60 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
56542024-11-28T02:05:23.560ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = ab660bac-1609-472c-99ad-75732ab64ce5 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
56552024-11-28T02:05:23.561ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = ab660bac-1609-472c-99ad-75732ab64ce5 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
56562024-11-28T02:05:23.563ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2487 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = ab660bac-1609-472c-99ad-75732ab64ce5 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
56572024-11-28T02:05:24.194ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1247 local_addr = [fd00:1122:3344:101::b]:55621 method = GET remote_addr = [fd00:1122:3344:101::d]:56417 req_id = 729fc844-4761-4d56-abfc-3f5a4597232c response_code = 200 uri = /2469a748-bcd2-4e0d-99cc-f79def0452cc
56582024-11-28T02:05:24.582ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = ac7a94ed-085e-4cf5-9d70-978ff6c58be2 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
56592024-11-28T02:05:24.583ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = ac7a94ed-085e-4cf5-9d70-978ff6c58be2 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
56602024-11-28T02:05:24.585ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2684 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = ac7a94ed-085e-4cf5-9d70-978ff6c58be2 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
56612024-11-28T02:05:25.604ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = e9d2b886-277d-4c32-adaf-a787e7c51fe2 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
56622024-11-28T02:05:25.604ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = e9d2b886-277d-4c32-adaf-a787e7c51fe2 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
56632024-11-28T02:05:25.606ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2286 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = e9d2b886-277d-4c32-adaf-a787e7c51fe2 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
56642024-11-28T02:05:26.625ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = a76d24c8-8596-40b2-90fe-bcb93c9bf122 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
56652024-11-28T02:05:26.625ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = a76d24c8-8596-40b2-90fe-bcb93c9bf122 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
56662024-11-28T02:05:26.627ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2372 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = a76d24c8-8596-40b2-90fe-bcb93c9bf122 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
56672024-11-28T02:05:27.647ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 235b99d7-aa66-4e5c-8a6d-eff3e4b6f31e roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
56682024-11-28T02:05:27.647ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 235b99d7-aa66-4e5c-8a6d-eff3e4b6f31e resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
56692024-11-28T02:05:27.649ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2338 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 235b99d7-aa66-4e5c-8a6d-eff3e4b6f31e response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
56702024-11-28T02:05:28.668ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = f6596d92-4e22-4f42-b5df-3cfd2de063ea roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
56712024-11-28T02:05:28.669ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = f6596d92-4e22-4f42-b5df-3cfd2de063ea resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
56722024-11-28T02:05:28.671ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2729 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = f6596d92-4e22-4f42-b5df-3cfd2de063ea response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
56732024-11-28T02:05:29.690ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 17292e20-89cf-4bff-8990-34e1af89ff22 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
56742024-11-28T02:05:29.690ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 17292e20-89cf-4bff-8990-34e1af89ff22 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
56752024-11-28T02:05:29.692ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2026 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 17292e20-89cf-4bff-8990-34e1af89ff22 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
56762024-11-28T02:05:30.479ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 75 reason = Timeout
56772024-11-28T02:05:30.479ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 75 reason = Timeout
56782024-11-28T02:05:30.479ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
56792024-11-28T02:05:30.479ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
56802024-11-28T02:05:30.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
56812024-11-28T02:05:30.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
56822024-11-28T02:05:30.481ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
56832024-11-28T02:05:30.481ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
56842024-11-28T02:05:30.481ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
56852024-11-28T02:05:30.484ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
56862024-11-28T02:05:30.484ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
56872024-11-28T02:05:30.484ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
56882024-11-28T02:05:30.486ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
56892024-11-28T02:05:30.486ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
56902024-11-28T02:05:30.487ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
56912024-11-28T02:05:30.490ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 67ac247f-c934-4c78-857a-fba4fbeb9937, lookup_type: ById(67ac247f-c934-4c78-857a-fba4fbeb9937) } result = Ok(())
56922024-11-28T02:05:30.490ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
56932024-11-28T02:05:30.490ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
56942024-11-28T02:05:30.501ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 67ac247f-c934-4c78-857a-fba4fbeb9937 original_time_created = 2024-11-28 01:53:03.159475 UTC target_id = 67ac247f-c934-4c78-857a-fba4fbeb9937 time_created = 2024-11-28 01:53:03.159475 UTC
56952024-11-28T02:05:30.501ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 22.207846ms iteration = 75 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"67ac247f-c934-4c78-857a-fba4fbeb9937","time_created":"2024-11-28T01:53:03.159475Z"}
56962024-11-28T02:05:30.510ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 30.927808ms iteration = 75 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
56972024-11-28T02:05:30.711ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 5347d24e-71c0-429d-8d28-a8586df363d1 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
56982024-11-28T02:05:30.711ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 5347d24e-71c0-429d-8d28-a8586df363d1 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
56992024-11-28T02:05:30.712ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1954 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 5347d24e-71c0-429d-8d28-a8586df363d1 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
57002024-11-28T02:05:31.732ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 903baefd-09bf-4ab9-a44b-9193ae931f4d roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
57012024-11-28T02:05:31.732ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 903baefd-09bf-4ab9-a44b-9193ae931f4d resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
57022024-11-28T02:05:31.735ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2826 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 903baefd-09bf-4ab9-a44b-9193ae931f4d response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
57032024-11-28T02:05:32.754ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 0e34b186-e774-4220-9a2f-4bf7d8dfc54d roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
57042024-11-28T02:05:32.755ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 0e34b186-e774-4220-9a2f-4bf7d8dfc54d resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
57052024-11-28T02:05:32.757ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2586 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 0e34b186-e774-4220-9a2f-4bf7d8dfc54d response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
57062024-11-28T02:05:33.776ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = dfebc6e1-d40f-4778-be21-245eade22e0e roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
57072024-11-28T02:05:33.776ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = dfebc6e1-d40f-4778-be21-245eade22e0e resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
57082024-11-28T02:05:33.778ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1914 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = dfebc6e1-d40f-4778-be21-245eade22e0e response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
57092024-11-28T02:05:34.195ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1776 local_addr = [fd00:1122:3344:101::b]:55621 method = GET remote_addr = [fd00:1122:3344:101::d]:56417 req_id = 87c0fe70-1ff5-4d63-a688-b195cf60de8b response_code = 200 uri = /2469a748-bcd2-4e0d-99cc-f79def0452cc
57102024-11-28T02:05:34.796ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 49780596-5d7e-42c3-b6d8-2fa1c5615cf8 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
57112024-11-28T02:05:34.796ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 49780596-5d7e-42c3-b6d8-2fa1c5615cf8 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
57122024-11-28T02:05:34.798ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2620 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 49780596-5d7e-42c3-b6d8-2fa1c5615cf8 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
57132024-11-28T02:05:35.818ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 8305c44c-af4e-425f-b250-55ad6fef2ccd roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
57142024-11-28T02:05:35.818ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 8305c44c-af4e-425f-b250-55ad6fef2ccd resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
57152024-11-28T02:05:35.820ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2508 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 8305c44c-af4e-425f-b250-55ad6fef2ccd response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
57162024-11-28T02:05:36.839ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = bf39b552-5416-4086-913b-79ec2ab24629 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
57172024-11-28T02:05:36.840ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = bf39b552-5416-4086-913b-79ec2ab24629 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
57182024-11-28T02:05:36.842ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2598 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = bf39b552-5416-4086-913b-79ec2ab24629 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
57192024-11-28T02:05:37.860ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 1421b705-36ea-4007-99f6-8b273ea6ea24 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
57202024-11-28T02:05:37.861ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 1421b705-36ea-4007-99f6-8b273ea6ea24 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
57212024-11-28T02:05:37.862ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2232 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 1421b705-36ea-4007-99f6-8b273ea6ea24 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
57222024-11-28T02:05:38.881ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 4350da0a-ed17-4ba1-8cb9-5c350a8c3c20 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
57232024-11-28T02:05:38.882ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 4350da0a-ed17-4ba1-8cb9-5c350a8c3c20 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
57242024-11-28T02:05:38.883ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2389 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 4350da0a-ed17-4ba1-8cb9-5c350a8c3c20 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
57252024-11-28T02:05:39.541ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:40402 req_id = 136668af-9198-4b25-b7e8-1aba2355c39d roles = RoleSet { roles: {} } uri = /nat/ipv4/changeset/10?limit=100
57262024-11-28T02:05:39.542ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:40402 req_id = 136668af-9198-4b25-b7e8-1aba2355c39d resource = Database result = Ok(()) uri = /nat/ipv4/changeset/10?limit=100
57272024-11-28T02:05:39.545ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 3887 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:40402 req_id = 136668af-9198-4b25-b7e8-1aba2355c39d response_code = 200 uri = /nat/ipv4/changeset/10?limit=100
57282024-11-28T02:05:39.903ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 70dea7dd-d95f-454f-86b7-0714048ea66a roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
57292024-11-28T02:05:39.904ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 70dea7dd-d95f-454f-86b7-0714048ea66a resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
57302024-11-28T02:05:39.906ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2350 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 70dea7dd-d95f-454f-86b7-0714048ea66a response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
57312024-11-28T02:05:39.989ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::b]:12221 remote_addr = [fd00:1122:3344:101::1]:44794
57322024-11-28T02:05:39.990ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::1]:44794 req_id = 13a6ad7a-735f-4d02-a8ae-e9c1713d459d roles = RoleSet { roles: {} } uri = /metrics/producers
57332024-11-28T02:05:39.990ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::1]:44794 req_id = 13a6ad7a-735f-4d02-a8ae-e9c1713d459d resource = Database result = Ok(()) uri = /metrics/producers
57342024-11-28T02:05:40.025ZINFOnexus: assigned collector to new producer collector_id = 119eed64-2e24-4d5f-8c42-be6cd8bec8b5 file = nexus/src/app/oximeter.rs:76 producer_id = 313847b3-d670-4167-9b09-9e7d3dfcb2a4
57352024-11-28T02:05:40.025ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 35600 local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::1]:44794 req_id = 13a6ad7a-735f-4d02-a8ae-e9c1713d459d response_code = 201 uri = /metrics/producers
57362024-11-28T02:05:40.479ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 28 reason = Timeout
57372024-11-28T02:05:40.479ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 26 reason = Timeout
57382024-11-28T02:05:40.479ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
57392024-11-28T02:05:40.479ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
57402024-11-28T02:05:40.479ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
57412024-11-28T02:05:40.479ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 26 reason = Timeout
57422024-11-28T02:05:40.479ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
57432024-11-28T02:05:40.479ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
57442024-11-28T02:05:40.479ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 76 reason = Timeout
57452024-11-28T02:05:40.479ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 26 reason = Timeout
57462024-11-28T02:05:40.479ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
57472024-11-28T02:05:40.479ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
57482024-11-28T02:05:40.479ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 26 reason = Timeout
57492024-11-28T02:05:40.479ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
57502024-11-28T02:05:40.479ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 26 reason = Timeout
57512024-11-28T02:05:40.479ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 26 reason = Timeout
57522024-11-28T02:05:40.479ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
57532024-11-28T02:05:40.479ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 26 reason = Timeout
57542024-11-28T02:05:40.479ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
57552024-11-28T02:05:40.479ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 30 reason = Timeout
57562024-11-28T02:05:40.479ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 26 reason = Timeout
57572024-11-28T02:05:40.479ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
57582024-11-28T02:05:40.479ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 26 reason = Timeout
57592024-11-28T02:05:40.479ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
57602024-11-28T02:05:40.479ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
57612024-11-28T02:05:40.479ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 76 reason = Timeout
57622024-11-28T02:05:40.479ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
57632024-11-28T02:05:40.479ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 27 reason = Timeout
57642024-11-28T02:05:40.479ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
57652024-11-28T02:05:40.479ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 26 reason = Timeout
57662024-11-28T02:05:40.479ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
57672024-11-28T02:05:40.479ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 26 reason = Timeout
57682024-11-28T02:05:40.479ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
57692024-11-28T02:05:40.479ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 26 reason = Timeout
57702024-11-28T02:05:40.479ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
57712024-11-28T02:05:40.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
57722024-11-28T02:05:40.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
57732024-11-28T02:05:40.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
57742024-11-28T02:05:40.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
57752024-11-28T02:05:40.480ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
57762024-11-28T02:05:40.480ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
57772024-11-28T02:05:40.480ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
57782024-11-28T02:05:40.480ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
57792024-11-28T02:05:40.480ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
57802024-11-28T02:05:40.480ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
57812024-11-28T02:05:40.480ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
57822024-11-28T02:05:40.480ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
57832024-11-28T02:05:40.480ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
57842024-11-28T02:05:40.480ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
57852024-11-28T02:05:40.480ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
57862024-11-28T02:05:40.480ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
57872024-11-28T02:05:40.481ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
57882024-11-28T02:05:40.482ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.425009ms iteration = 26 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
57892024-11-28T02:05:40.483ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
57902024-11-28T02:05:40.483ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
57912024-11-28T02:05:40.483ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
57922024-11-28T02:05:40.485ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
57932024-11-28T02:05:40.487ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
57942024-11-28T02:05:40.487ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
57952024-11-28T02:05:40.487ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
57962024-11-28T02:05:40.487ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-313847b3-d670-4167-9b09-9e7d3dfcb2a4.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-313847b3-d670-4167-9b09-9e7d3dfcb2a4.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1606, tv_nsec: 957426255 } })
57972024-11-28T02:05:40.487ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1103
57982024-11-28T02:05:40.490ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-313847b3-d670-4167-9b09-9e7d3dfcb2a4.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-313847b3-d670-4167-9b09-9e7d3dfcb2a4.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1606, tv_nsec: 959912268 } })
57992024-11-28T02:05:40.490ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1103
58002024-11-28T02:05:40.492ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
58012024-11-28T02:05:40.495ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
58022024-11-28T02:05:40.495ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
58032024-11-28T02:05:40.495ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
58042024-11-28T02:05:40.531ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 51.617436ms iteration = 26 status = {"errors":[],"garbage_collect_requested":[]}
58052024-11-28T02:05:40.532ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
58062024-11-28T02:05:40.534ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
58072024-11-28T02:05:40.534ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
58082024-11-28T02:05:40.534ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
58092024-11-28T02:05:40.534ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
58102024-11-28T02:05:40.535ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
58112024-11-28T02:05:40.535ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
58122024-11-28T02:05:40.535ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
58132024-11-28T02:05:40.536ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
58142024-11-28T02:05:40.537ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
58152024-11-28T02:05:40.538ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
58162024-11-28T02:05:40.538ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 59.088097ms iteration = 26 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
58172024-11-28T02:05:40.538ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
58182024-11-28T02:05:40.538ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 59.316541ms iteration = 26 status = {"errors":[],"requests_created_ok":[],"start_invoked_ok":[]}
58192024-11-28T02:05:40.539ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
58202024-11-28T02:05:40.539ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
58212024-11-28T02:05:40.539ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
58222024-11-28T02:05:40.540ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
58232024-11-28T02:05:40.540ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
58242024-11-28T02:05:40.540ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 61.523329ms iteration = 26 status = {"errors":[],"records_set_to_done":[]}
58252024-11-28T02:05:40.547ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
58262024-11-28T02:05:40.549ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
58272024-11-28T02:05:40.549ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
58282024-11-28T02:05:40.549ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
58292024-11-28T02:05:40.549ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
58302024-11-28T02:05:40.550ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
58312024-11-28T02:05:40.551ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
58322024-11-28T02:05:40.551ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
58332024-11-28T02:05:40.552ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
58342024-11-28T02:05:40.552ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
58352024-11-28T02:05:40.552ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
58362024-11-28T02:05:40.552ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
58372024-11-28T02:05:40.552ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
58382024-11-28T02:05:40.552ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
58392024-11-28T02:05:40.552ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
58402024-11-28T02:05:40.553ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
58412024-11-28T02:05:40.553ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 74.25578ms iteration = 26 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
58422024-11-28T02:05:40.554ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
58432024-11-28T02:05:40.555ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
58442024-11-28T02:05:40.555ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
58452024-11-28T02:05:40.555ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-313847b3-d670-4167-9b09-9e7d3dfcb2a4.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-313847b3-d670-4167-9b09-9e7d3dfcb2a4.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1607, tv_nsec: 25512417 } })
58462024-11-28T02:05:40.555ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1103 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc
58472024-11-28T02:05:40.556ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
58482024-11-28T02:05:40.557ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
58492024-11-28T02:05:40.557ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
58502024-11-28T02:05:40.557ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
58512024-11-28T02:05:40.559ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
58522024-11-28T02:05:40.559ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
58532024-11-28T02:05:40.559ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 67ac247f-c934-4c78-857a-fba4fbeb9937, lookup_type: ById(67ac247f-c934-4c78-857a-fba4fbeb9937) } result = Ok(())
58542024-11-28T02:05:40.559ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
58552024-11-28T02:05:40.559ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
58562024-11-28T02:05:40.559ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "37257c75-fe34-421a-8576-a6d7be58428a", "content-length": "26", "date": "Thu, 28 Nov 2024 02:05:40 GMT"} })
58572024-11-28T02:05:40.559ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
58582024-11-28T02:05:40.562ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
58592024-11-28T02:05:40.562ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
58602024-11-28T02:05:40.562ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
58612024-11-28T02:05:40.562ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "9958f4b9-83d4-49eb-9473-c295224b20cb", "content-length": "26", "date": "Thu, 28 Nov 2024 02:05:40 GMT"} })
58622024-11-28T02:05:40.562ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
58632024-11-28T02:05:40.564ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
58642024-11-28T02:05:40.566ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
58652024-11-28T02:05:40.566ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
58662024-11-28T02:05:40.568ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
58672024-11-28T02:05:40.568ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 67ac247f-c934-4c78-857a-fba4fbeb9937 original_time_created = 2024-11-28 01:53:03.159475 UTC target_id = 67ac247f-c934-4c78-857a-fba4fbeb9937 time_created = 2024-11-28 01:53:03.159475 UTC
58682024-11-28T02:05:40.568ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 89.068157ms iteration = 76 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"67ac247f-c934-4c78-857a-fba4fbeb9937","time_created":"2024-11-28T01:53:03.159475Z"}
58692024-11-28T02:05:40.568ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
58702024-11-28T02:05:40.570ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
58712024-11-28T02:05:40.571ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
58722024-11-28T02:05:40.571ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
58732024-11-28T02:05:40.574ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
58742024-11-28T02:05:40.574ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
58752024-11-28T02:05:40.574ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
58762024-11-28T02:05:40.574ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
58772024-11-28T02:05:40.574ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 313847b3-d670-4167-9b09-9e7d3dfcb2a4, lookup_type: ById(313847b3-d670-4167-9b09-9e7d3dfcb2a4) } result = Ok(())
58782024-11-28T02:05:40.574ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
58792024-11-28T02:05:40.574ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 95.409397ms iteration = 76 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
58802024-11-28T02:05:40.575ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
58812024-11-28T02:05:40.576ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 97.266609ms iteration = 26 status = 0
58822024-11-28T02:05:40.600ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 121.240716ms iteration = 26 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[]}
58832024-11-28T02:05:40.603ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
58842024-11-28T02:05:40.603ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
58852024-11-28T02:05:40.603ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 124.624455ms iteration = 26 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
58862024-11-28T02:05:40.619ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
58872024-11-28T02:05:40.619ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
58882024-11-28T02:05:40.654ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
58892024-11-28T02:05:40.654ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
58902024-11-28T02:05:40.656ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
58912024-11-28T02:05:40.658ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 178.711133ms iteration = 30 status = {"physical_disks_added":0}
58922024-11-28T02:05:40.658ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
58932024-11-28T02:05:40.658ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
58942024-11-28T02:05:40.658ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
58952024-11-28T02:05:40.660ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
58962024-11-28T02:05:40.661ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
58972024-11-28T02:05:40.661ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
58982024-11-28T02:05:40.661ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
58992024-11-28T02:05:40.781ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 301.67847ms iteration = 26 status = {}
59002024-11-28T02:05:40.793ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1111 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc zone_address = fd00:1122:3344:101::2
59012024-11-28T02:05:40.793ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
59022024-11-28T02:05:40.794ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "b3c65840-12ac-4b85-9f02-9b1c97f791b8", "content-length": "26", "date": "Thu, 28 Nov 2024 02:05:40 GMT"} })
59032024-11-28T02:05:40.794ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1116 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
59042024-11-28T02:05:40.796ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc
59052024-11-28T02:05:40.804ZDEBGnexus (BackgroundTasks): client request SledAgent = 313847b3-d670-4167-9b09-9e7d3dfcb2a4 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
59062024-11-28T02:05:40.804ZDEBGnexus (BackgroundTasks): client response SledAgent = 313847b3-d670-4167-9b09-9e7d3dfcb2a4 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "6237e0a9-2bc4-4e97-b1e9-af5f642172e4", "content-length": "691", "date": "Thu, 28 Nov 2024 02:05:40 GMT"} })
59072024-11-28T02:05:40.804ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
59082024-11-28T02:05:40.804ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
59092024-11-28T02:05:40.804ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
59102024-11-28T02:05:40.806ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 326.513352ms iteration = 27 status = {}
59112024-11-28T02:05:40.806ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
59122024-11-28T02:05:40.806ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "fb1f160b-217a-4126-9f71-f5198ef6ca63", "content-length": "2", "date": "Thu, 28 Nov 2024 02:05:40 GMT"} })
59132024-11-28T02:05:40.807ZINFOnexus (BackgroundTasks): VPC route manager sled 313847b3-d670-4167-9b09-9e7d3dfcb2a4 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
59142024-11-28T02:05:40.807ZDEBGnexus (BackgroundTasks): client request SledAgent = 313847b3-d670-4167-9b09-9e7d3dfcb2a4 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
59152024-11-28T02:05:40.807ZDEBGnexus (BackgroundTasks): client response SledAgent = 313847b3-d670-4167-9b09-9e7d3dfcb2a4 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "498b768b-9356-4b2f-8086-7f6c281612a3", "content-length": "372", "date": "Thu, 28 Nov 2024 02:05:40 GMT"} })
59162024-11-28T02:05:40.807ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
59172024-11-28T02:05:40.807ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
59182024-11-28T02:05:40.808ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
59192024-11-28T02:05:40.808ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
59202024-11-28T02:05:40.810ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 331.445127ms iteration = 26 status = null
59212024-11-28T02:05:40.845ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
59222024-11-28T02:05:40.845ZDEBGnexus (BackgroundTasks): client request SledAgent = 313847b3-d670-4167-9b09-9e7d3dfcb2a4 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
59232024-11-28T02:05:40.846ZDEBGnexus (BackgroundTasks): client response SledAgent = 313847b3-d670-4167-9b09-9e7d3dfcb2a4 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "5769d0c3-82c4-4c9f-a4a1-7a6b626b7fd1", "date": "Thu, 28 Nov 2024 02:05:40 GMT"} })
59242024-11-28T02:05:40.846ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
59252024-11-28T02:05:40.846ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
59262024-11-28T02:05:40.848ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
59272024-11-28T02:05:40.848ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
59282024-11-28T02:05:40.850ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
59292024-11-28T02:05:40.850ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
59302024-11-28T02:05:40.853ZINFOnexus (BackgroundTasks): VPC route manager sled 313847b3-d670-4167-9b09-9e7d3dfcb2a4 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
59312024-11-28T02:05:40.853ZDEBGnexus (BackgroundTasks): client request SledAgent = 313847b3-d670-4167-9b09-9e7d3dfcb2a4 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
59322024-11-28T02:05:40.892ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
59332024-11-28T02:05:40.892ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
59342024-11-28T02:05:40.894ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 4bb2986f-2e92-4ac3-9dc1-591b6bb60b3e, rack_id: 92a8ff6e-21d3-4c28-bcad-e323847985dc, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(ea300318-9d62-42ed-80ea-21c86590a166) } rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc switch_location = Switch0
59352024-11-28T02:05:40.894ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
59362024-11-28T02:05:40.894ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
59372024-11-28T02:05:40.901ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
59382024-11-28T02:05:40.902ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "e53d629c-8de7-4979-a8dd-1d5f9d705079", "content-length": "134", "date": "Thu, 28 Nov 2024 02:05:40 GMT"} })
59392024-11-28T02:05:40.902ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1746 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
59402024-11-28T02:05:40.902ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1773 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = ea300318-9d62-42ed-80ea-21c86590a166
59412024-11-28T02:05:40.902ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
59422024-11-28T02:05:40.902ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
59432024-11-28T02:05:40.902ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
59442024-11-28T02:05:40.902ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
59452024-11-28T02:05:40.902ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
59462024-11-28T02:05:40.902ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1920 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
59472024-11-28T02:05:40.902ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
59482024-11-28T02:05:40.902ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
59492024-11-28T02:05:40.902ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc
59502024-11-28T02:05:40.902ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
59512024-11-28T02:05:40.902ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
59522024-11-28T02:05:40.904ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
59532024-11-28T02:05:40.905ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "50b65b83-338b-4078-b2ad-b21ac0de7827", "content-length": "2", "date": "Thu, 28 Nov 2024 02:05:40 GMT"} })
59542024-11-28T02:05:40.905ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
59552024-11-28T02:05:40.905ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "32f82060-72a8-4b52-b33b-56d669898a1d", "content-length": "52", "date": "Thu, 28 Nov 2024 02:05:40 GMT"} })
59562024-11-28T02:05:40.905ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc switch_location = Switch0
59572024-11-28T02:05:40.905ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
59582024-11-28T02:05:40.940ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 20a6d818-9fe1-4116-aebb-f048ef29b935 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
59592024-11-28T02:05:40.941ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 20a6d818-9fe1-4116-aebb-f048ef29b935 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
59602024-11-28T02:05:40.942ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1806 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 20a6d818-9fe1-4116-aebb-f048ef29b935 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
59612024-11-28T02:05:40.950ZDEBGnexus (BackgroundTasks): client response SledAgent = 313847b3-d670-4167-9b09-9e7d3dfcb2a4 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "0d907eed-3194-478c-a667-234e37f632ff", "date": "Thu, 28 Nov 2024 02:05:40 GMT"} })
59622024-11-28T02:05:40.952ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 472.774022ms iteration = 28 status = {}
59632024-11-28T02:05:41.054ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "f7c6ba18-06fc-4d77-a414-b021cb2b8653", "date": "Thu, 28 Nov 2024 02:05:40 GMT"} })
59642024-11-28T02:05:41.054ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc switch = Switch0
59652024-11-28T02:05:41.054ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
59662024-11-28T02:05:41.054ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "cafa8c0c-3e1b-463a-b96c-17f84ebb8cbb", "content-length": "682", "date": "Thu, 28 Nov 2024 02:05:40 GMT"} })
59672024-11-28T02:05:41.054ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
59682024-11-28T02:05:41.055ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
59692024-11-28T02:05:41.059ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
59702024-11-28T02:05:41.059ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
59712024-11-28T02:05:41.062ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
59722024-11-28T02:05:41.062ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
59732024-11-28T02:05:41.064ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
59742024-11-28T02:05:41.064ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
59752024-11-28T02:05:41.071ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 592.43244ms iteration = 26 status = {}
59762024-11-28T02:05:41.890ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::b]:12221 remote_addr = [fd00:1122:3344:101::2]:44458
59772024-11-28T02:05:41.890ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:44458 req_id = 57a0526a-1970-4d71-ac85-f2631565776f roles = RoleSet { roles: {} } uri = /metrics/producers
59782024-11-28T02:05:41.890ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:44458 req_id = 57a0526a-1970-4d71-ac85-f2631565776f resource = Database result = Ok(()) uri = /metrics/producers
59792024-11-28T02:05:41.929ZINFOnexus: assigned collector to new producer collector_id = 119eed64-2e24-4d5f-8c42-be6cd8bec8b5 file = nexus/src/app/oximeter.rs:76 producer_id = 4ea85f4f-c0ac-4d9e-ae1a-e4f649a8809c
59802024-11-28T02:05:41.929ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 39202 local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:44458 req_id = 57a0526a-1970-4d71-ac85-f2631565776f response_code = 201 uri = /metrics/producers
59812024-11-28T02:05:41.967ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 1be38eaf-28a5-4bc5-8195-58f05474c47d roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
59822024-11-28T02:05:41.967ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 1be38eaf-28a5-4bc5-8195-58f05474c47d resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
59832024-11-28T02:05:41.969ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2225 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 1be38eaf-28a5-4bc5-8195-58f05474c47d response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
59842024-11-28T02:05:42.123ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::b]:12221 remote_addr = [fd00:1122:3344:101::1]:58347
59852024-11-28T02:05:42.123ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::1]:58347 req_id = 6b01b9d2-b742-4af4-af75-71bc84891d93 roles = RoleSet { roles: {} } uri = /metrics/producers
59862024-11-28T02:05:42.123ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::1]:58347 req_id = 6b01b9d2-b742-4af4-af75-71bc84891d93 resource = Database result = Ok(()) uri = /metrics/producers
59872024-11-28T02:05:42.162ZINFOnexus: assigned collector to new producer collector_id = 119eed64-2e24-4d5f-8c42-be6cd8bec8b5 file = nexus/src/app/oximeter.rs:76 producer_id = 7d8f995c-f158-4cd9-9fae-a9ac17bfe5ed
59882024-11-28T02:05:42.162ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 38919 local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::1]:58347 req_id = 6b01b9d2-b742-4af4-af75-71bc84891d93 response_code = 201 uri = /metrics/producers
59892024-11-28T02:05:42.704ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::b]:12221 remote_addr = [fd00:1122:3344:101::2]:60199
59902024-11-28T02:05:42.704ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:60199 req_id = 7122f9e8-85a1-48a1-a5a9-eda4a575ba16 roles = RoleSet { roles: {} } uri = /metrics/producers
59912024-11-28T02:05:42.704ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:60199 req_id = 7122f9e8-85a1-48a1-a5a9-eda4a575ba16 resource = Database result = Ok(()) uri = /metrics/producers
59922024-11-28T02:05:42.740ZINFOnexus: assigned collector to new producer collector_id = 119eed64-2e24-4d5f-8c42-be6cd8bec8b5 file = nexus/src/app/oximeter.rs:76 producer_id = 37e16e5f-8832-402e-b7dd-2f5b87405438
59932024-11-28T02:05:42.740ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 36553 local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:60199 req_id = 7122f9e8-85a1-48a1-a5a9-eda4a575ba16 response_code = 201 uri = /metrics/producers
59942024-11-28T02:05:42.988ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = d87f9b6a-0b78-4565-99ee-e3324650923a roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
59952024-11-28T02:05:42.988ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = d87f9b6a-0b78-4565-99ee-e3324650923a resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
59962024-11-28T02:05:42.990ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2251 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = d87f9b6a-0b78-4565-99ee-e3324650923a response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
59972024-11-28T02:05:44.009ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 98312a64-de0e-4a4e-a6bd-b7fc20aaf138 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
59982024-11-28T02:05:44.010ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 98312a64-de0e-4a4e-a6bd-b7fc20aaf138 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
59992024-11-28T02:05:44.012ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2579 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 98312a64-de0e-4a4e-a6bd-b7fc20aaf138 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
60002024-11-28T02:05:44.195ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1788 local_addr = [fd00:1122:3344:101::b]:55621 method = GET remote_addr = [fd00:1122:3344:101::d]:56417 req_id = 5bdfc746-8b92-45e1-a3e3-508fabdce44d response_code = 200 uri = /2469a748-bcd2-4e0d-99cc-f79def0452cc
60012024-11-28T02:05:45.031ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = c8481289-bd42-452d-86fc-cc5ea76564f6 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
60022024-11-28T02:05:45.031ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = c8481289-bd42-452d-86fc-cc5ea76564f6 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
60032024-11-28T02:05:45.032ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1901 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = c8481289-bd42-452d-86fc-cc5ea76564f6 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
60042024-11-28T02:05:46.051ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = ad8340a5-9b60-4a40-ab28-f24b7bc6e296 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
60052024-11-28T02:05:46.052ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = ad8340a5-9b60-4a40-ab28-f24b7bc6e296 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
60062024-11-28T02:05:46.054ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 3020 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = ad8340a5-9b60-4a40-ab28-f24b7bc6e296 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
60072024-11-28T02:05:46.932ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::b]:12221 remote_addr = [fd00:1122:3344:101::2]:60551
60082024-11-28T02:05:46.932ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:60551 req_id = f86cce66-8265-471b-8b05-ac2eef62cb09 roles = RoleSet { roles: {} } uri = /metrics/producers
60092024-11-28T02:05:46.932ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:60551 req_id = f86cce66-8265-471b-8b05-ac2eef62cb09 resource = Database result = Ok(()) uri = /metrics/producers
60102024-11-28T02:05:46.943ZINFOnexus: assigned collector to new producer collector_id = 119eed64-2e24-4d5f-8c42-be6cd8bec8b5 file = nexus/src/app/oximeter.rs:76 producer_id = 27b44fd9-9dc4-42a5-82e8-5084c0434475
60112024-11-28T02:05:46.943ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 10715 local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:60551 req_id = f86cce66-8265-471b-8b05-ac2eef62cb09 response_code = 201 uri = /metrics/producers
60122024-11-28T02:05:47.073ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = f84a6306-fef1-40f2-a31b-ee5aaa688cf9 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
60132024-11-28T02:05:47.073ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = f84a6306-fef1-40f2-a31b-ee5aaa688cf9 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
60142024-11-28T02:05:47.077ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 4637 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = f84a6306-fef1-40f2-a31b-ee5aaa688cf9 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
60152024-11-28T02:05:48.098ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 5d375208-1c2f-4b7c-9e9e-51eca451f7fe roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
60162024-11-28T02:05:48.098ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 5d375208-1c2f-4b7c-9e9e-51eca451f7fe resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
60172024-11-28T02:05:48.100ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2398 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 5d375208-1c2f-4b7c-9e9e-51eca451f7fe response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
60182024-11-28T02:05:49.119ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = aac3b5f4-9d92-4858-876e-072a28795674 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
60192024-11-28T02:05:49.119ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = aac3b5f4-9d92-4858-876e-072a28795674 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
60202024-11-28T02:05:49.121ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1931 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = aac3b5f4-9d92-4858-876e-072a28795674 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
60212024-11-28T02:05:50.140ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = ee805e07-8c3f-440a-86c9-e1a58da92339 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
60222024-11-28T02:05:50.140ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = ee805e07-8c3f-440a-86c9-e1a58da92339 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
60232024-11-28T02:05:50.142ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2305 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = ee805e07-8c3f-440a-86c9-e1a58da92339 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
60242024-11-28T02:05:50.478ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 77 reason = Timeout
60252024-11-28T02:05:50.479ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 77 reason = Timeout
60262024-11-28T02:05:50.479ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
60272024-11-28T02:05:50.479ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
60282024-11-28T02:05:50.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
60292024-11-28T02:05:50.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
60302024-11-28T02:05:50.481ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
60312024-11-28T02:05:50.481ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
60322024-11-28T02:05:50.481ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
60332024-11-28T02:05:50.484ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
60342024-11-28T02:05:50.484ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
60352024-11-28T02:05:50.484ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
60362024-11-28T02:05:50.486ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
60372024-11-28T02:05:50.486ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
60382024-11-28T02:05:50.488ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
60392024-11-28T02:05:50.491ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 67ac247f-c934-4c78-857a-fba4fbeb9937, lookup_type: ById(67ac247f-c934-4c78-857a-fba4fbeb9937) } result = Ok(())
60402024-11-28T02:05:50.491ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
60412024-11-28T02:05:50.491ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
60422024-11-28T02:05:50.501ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 67ac247f-c934-4c78-857a-fba4fbeb9937 original_time_created = 2024-11-28 01:53:03.159475 UTC target_id = 67ac247f-c934-4c78-857a-fba4fbeb9937 time_created = 2024-11-28 01:53:03.159475 UTC
60432024-11-28T02:05:50.501ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 22.712674ms iteration = 77 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"67ac247f-c934-4c78-857a-fba4fbeb9937","time_created":"2024-11-28T01:53:03.159475Z"}
60442024-11-28T02:05:50.509ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 30.065813ms iteration = 77 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
60452024-11-28T02:05:51.160ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = c44032a3-334a-4eaf-baaf-df57fcba4d7a roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
60462024-11-28T02:05:51.161ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = c44032a3-334a-4eaf-baaf-df57fcba4d7a resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
60472024-11-28T02:05:51.163ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2492 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = c44032a3-334a-4eaf-baaf-df57fcba4d7a response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
60482024-11-28T02:05:52.183ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 3cfb66a7-2927-4742-9905-e7f2c7eba769 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
60492024-11-28T02:05:52.183ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 3cfb66a7-2927-4742-9905-e7f2c7eba769 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
60502024-11-28T02:05:52.185ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2460 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 3cfb66a7-2927-4742-9905-e7f2c7eba769 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
60512024-11-28T02:05:53.204ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 32459ca1-f599-4308-befa-3669cb29a15d roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
60522024-11-28T02:05:53.205ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 32459ca1-f599-4308-befa-3669cb29a15d resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
60532024-11-28T02:05:53.207ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2256 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 32459ca1-f599-4308-befa-3669cb29a15d response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
60542024-11-28T02:05:54.186ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:55495 req_id = 7d340a2a-c48a-4391-a2c9-f31d7c6b84a7 roles = RoleSet { roles: {} } uri = /metrics/collectors/119eed64-2e24-4d5f-8c42-be6cd8bec8b5/producers?sort_by=id_ascending
60552024-11-28T02:05:54.186ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:55495 req_id = 7d340a2a-c48a-4391-a2c9-f31d7c6b84a7 resource = Database result = Ok(()) uri = /metrics/collectors/119eed64-2e24-4d5f-8c42-be6cd8bec8b5/producers?sort_by=id_ascending
60562024-11-28T02:05:54.189ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 3232 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:55495 req_id = 7d340a2a-c48a-4391-a2c9-f31d7c6b84a7 response_code = 200 uri = /metrics/collectors/119eed64-2e24-4d5f-8c42-be6cd8bec8b5/producers?sort_by=id_ascending
60572024-11-28T02:05:54.189ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:55495 req_id = 9980f394-43ee-4ba3-9b7e-2fbd85ce0309 roles = RoleSet { roles: {} } uri = /metrics/collectors/119eed64-2e24-4d5f-8c42-be6cd8bec8b5/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJjNDM0YjQ0OC01MTk5LTQ2NmUtYWYzMi00ZjYyOWUzMWM3YjEifX0%3D
60582024-11-28T02:05:54.189ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:55495 req_id = 9980f394-43ee-4ba3-9b7e-2fbd85ce0309 resource = Database result = Ok(()) uri = /metrics/collectors/119eed64-2e24-4d5f-8c42-be6cd8bec8b5/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJjNDM0YjQ0OC01MTk5LTQ2NmUtYWYzMi00ZjYyOWUzMWM3YjEifX0%3D
60592024-11-28T02:05:54.192ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2559 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:55495 req_id = 9980f394-43ee-4ba3-9b7e-2fbd85ce0309 response_code = 200 uri = /metrics/collectors/119eed64-2e24-4d5f-8c42-be6cd8bec8b5/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJjNDM0YjQ0OC01MTk5LTQ2NmUtYWYzMi00ZjYyOWUzMWM3YjEifX0%3D
60602024-11-28T02:05:54.194ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1228 local_addr = [fd00:1122:3344:101::b]:55621 method = GET remote_addr = [fd00:1122:3344:101::d]:56417 req_id = eb13db0e-a7c5-42c2-9f66-04c6a683e981 response_code = 200 uri = /2469a748-bcd2-4e0d-99cc-f79def0452cc
60612024-11-28T02:05:54.225ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 53d2b30c-50dd-41d1-a2ae-cc43b04bfedf roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
60622024-11-28T02:05:54.226ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 53d2b30c-50dd-41d1-a2ae-cc43b04bfedf resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
60632024-11-28T02:05:54.227ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2106 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 53d2b30c-50dd-41d1-a2ae-cc43b04bfedf response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
60642024-11-28T02:05:55.247ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 5f78e95d-fb7b-469a-8e70-f92af5da5949 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
60652024-11-28T02:05:55.248ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 5f78e95d-fb7b-469a-8e70-f92af5da5949 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
60662024-11-28T02:05:55.250ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2539 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 5f78e95d-fb7b-469a-8e70-f92af5da5949 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
60672024-11-28T02:05:56.272ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 97e38542-bfd3-47fb-aa1d-4a7bc4f5c6e4 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
60682024-11-28T02:05:56.272ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 97e38542-bfd3-47fb-aa1d-4a7bc4f5c6e4 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
60692024-11-28T02:05:56.274ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2287 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 97e38542-bfd3-47fb-aa1d-4a7bc4f5c6e4 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
60702024-11-28T02:05:56.874ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::b]:12221 remote_addr = [fd00:1122:3344:101::2]:39915
60712024-11-28T02:05:56.874ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:39915 req_id = 054dfd20-6390-4be7-9d51-dc3d58fe21d1 roles = RoleSet { roles: {} } uri = /metrics/producers
60722024-11-28T02:05:56.875ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:39915 req_id = 054dfd20-6390-4be7-9d51-dc3d58fe21d1 resource = Database result = Ok(()) uri = /metrics/producers
60732024-11-28T02:05:56.887ZINFOnexus: assigned collector to new producer collector_id = 119eed64-2e24-4d5f-8c42-be6cd8bec8b5 file = nexus/src/app/oximeter.rs:76 producer_id = c434b448-5199-466e-af32-4f629e31c7b1
60742024-11-28T02:05:56.887ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 12634 local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:39915 req_id = 054dfd20-6390-4be7-9d51-dc3d58fe21d1 response_code = 201 uri = /metrics/producers
60752024-11-28T02:05:57.293ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 8a72ea66-c23a-45a8-874d-21ee8c2e796b roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
60762024-11-28T02:05:57.294ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 8a72ea66-c23a-45a8-874d-21ee8c2e796b resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
60772024-11-28T02:05:57.296ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2480 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 8a72ea66-c23a-45a8-874d-21ee8c2e796b response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
60782024-11-28T02:05:58.315ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = c4b213f8-e3fb-400a-850a-96c144f7f33f roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
60792024-11-28T02:05:58.316ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = c4b213f8-e3fb-400a-850a-96c144f7f33f resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
60802024-11-28T02:05:58.318ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2348 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = c4b213f8-e3fb-400a-850a-96c144f7f33f response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
60812024-11-28T02:05:59.338ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = bd8a6a4a-522c-4c6c-a47f-8d998bd76d9f roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
60822024-11-28T02:05:59.338ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = bd8a6a4a-522c-4c6c-a47f-8d998bd76d9f resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
60832024-11-28T02:05:59.340ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2354 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = bd8a6a4a-522c-4c6c-a47f-8d998bd76d9f response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
60842024-11-28T02:06:00.358ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = af2b4c5c-e09a-4ae7-8735-1a336f420680 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
60852024-11-28T02:06:00.359ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = af2b4c5c-e09a-4ae7-8735-1a336f420680 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
60862024-11-28T02:06:00.360ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2156 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = af2b4c5c-e09a-4ae7-8735-1a336f420680 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
60872024-11-28T02:06:00.479ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 78 reason = Timeout
60882024-11-28T02:06:00.479ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 78 reason = Timeout
60892024-11-28T02:06:00.479ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
60902024-11-28T02:06:00.479ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
60912024-11-28T02:06:00.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
60922024-11-28T02:06:00.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
60932024-11-28T02:06:00.481ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
60942024-11-28T02:06:00.481ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
60952024-11-28T02:06:00.481ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
60962024-11-28T02:06:00.483ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.898248ms iteration = 78 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
60972024-11-28T02:06:00.483ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
60982024-11-28T02:06:00.483ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
60992024-11-28T02:06:00.483ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
61002024-11-28T02:06:00.485ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
61012024-11-28T02:06:00.485ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
61022024-11-28T02:06:00.487ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
61032024-11-28T02:06:00.489ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 67ac247f-c934-4c78-857a-fba4fbeb9937, lookup_type: ById(67ac247f-c934-4c78-857a-fba4fbeb9937) } result = Ok(())
61042024-11-28T02:06:00.489ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
61052024-11-28T02:06:00.489ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
61062024-11-28T02:06:00.500ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 67ac247f-c934-4c78-857a-fba4fbeb9937 original_time_created = 2024-11-28 01:53:03.159475 UTC target_id = 67ac247f-c934-4c78-857a-fba4fbeb9937 time_created = 2024-11-28 01:53:03.159475 UTC
61072024-11-28T02:06:00.500ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 21.008845ms iteration = 78 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"67ac247f-c934-4c78-857a-fba4fbeb9937","time_created":"2024-11-28T01:53:03.159475Z"}
61082024-11-28T02:06:01.380ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 1c7002d4-7600-438f-bc59-ba0342fc666f roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
61092024-11-28T02:06:01.380ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 1c7002d4-7600-438f-bc59-ba0342fc666f resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
61102024-11-28T02:06:01.382ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2321 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 1c7002d4-7600-438f-bc59-ba0342fc666f response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
61112024-11-28T02:06:02.402ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 8c929c02-5105-4adf-8d78-eb1ab8191fd9 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
61122024-11-28T02:06:02.402ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 8c929c02-5105-4adf-8d78-eb1ab8191fd9 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
61132024-11-28T02:06:02.404ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2379 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 8c929c02-5105-4adf-8d78-eb1ab8191fd9 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
61142024-11-28T02:06:03.423ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = ac5334e1-11b7-4c34-bad4-92b4c36eccdb roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
61152024-11-28T02:06:03.423ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = ac5334e1-11b7-4c34-bad4-92b4c36eccdb resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
61162024-11-28T02:06:03.425ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2662 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = ac5334e1-11b7-4c34-bad4-92b4c36eccdb response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
61172024-11-28T02:06:04.195ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1295 local_addr = [fd00:1122:3344:101::b]:55621 method = GET remote_addr = [fd00:1122:3344:101::d]:56417 req_id = 3034d85f-8b9b-4e35-b33e-0fe3987f1e84 response_code = 200 uri = /2469a748-bcd2-4e0d-99cc-f79def0452cc
61182024-11-28T02:06:04.445ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 9b40a867-0465-4305-a20b-85db83cf3bc7 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
61192024-11-28T02:06:04.445ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 9b40a867-0465-4305-a20b-85db83cf3bc7 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
61202024-11-28T02:06:04.446ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2003 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 9b40a867-0465-4305-a20b-85db83cf3bc7 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
61212024-11-28T02:06:05.466ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 64c87885-62b8-4ce2-93ff-48f852afe1d7 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
61222024-11-28T02:06:05.466ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 64c87885-62b8-4ce2-93ff-48f852afe1d7 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
61232024-11-28T02:06:05.468ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2256 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 64c87885-62b8-4ce2-93ff-48f852afe1d7 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
61242024-11-28T02:06:06.486ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 9bc6eee8-41c4-4a5a-a1b9-d9fe0a112df8 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
61252024-11-28T02:06:06.487ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 9bc6eee8-41c4-4a5a-a1b9-d9fe0a112df8 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
61262024-11-28T02:06:06.489ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2743 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 9bc6eee8-41c4-4a5a-a1b9-d9fe0a112df8 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
61272024-11-28T02:06:07.508ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 123f16c3-5476-4c13-8de2-23c94fff5560 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
61282024-11-28T02:06:07.508ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 123f16c3-5476-4c13-8de2-23c94fff5560 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
61292024-11-28T02:06:07.510ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2740 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 123f16c3-5476-4c13-8de2-23c94fff5560 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
61302024-11-28T02:06:08.531ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 34ecdd65-05e2-402f-94d7-5cbb606ce8c6 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
61312024-11-28T02:06:08.531ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 34ecdd65-05e2-402f-94d7-5cbb606ce8c6 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
61322024-11-28T02:06:08.533ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2285 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 34ecdd65-05e2-402f-94d7-5cbb606ce8c6 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
61332024-11-28T02:06:08.575ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:40402 req_id = 3d213a67-8f6a-45ca-8386-649119fbeccc roles = RoleSet { roles: {} } uri = /nat/ipv4/changeset/10?limit=100
61342024-11-28T02:06:08.576ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:40402 req_id = 3d213a67-8f6a-45ca-8386-649119fbeccc resource = Database result = Ok(()) uri = /nat/ipv4/changeset/10?limit=100
61352024-11-28T02:06:08.579ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 3606 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:40402 req_id = 3d213a67-8f6a-45ca-8386-649119fbeccc response_code = 200 uri = /nat/ipv4/changeset/10?limit=100
61362024-11-28T02:06:09.186ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:55495 req_id = 90ccf16f-3171-4c6c-9848-6cdbfbc2b750 roles = RoleSet { roles: {} } uri = /metrics/collectors/119eed64-2e24-4d5f-8c42-be6cd8bec8b5/producers?sort_by=id_ascending
61372024-11-28T02:06:09.187ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:55495 req_id = 90ccf16f-3171-4c6c-9848-6cdbfbc2b750 resource = Database result = Ok(()) uri = /metrics/collectors/119eed64-2e24-4d5f-8c42-be6cd8bec8b5/producers?sort_by=id_ascending
61382024-11-28T02:06:09.189ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 3129 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:55495 req_id = 90ccf16f-3171-4c6c-9848-6cdbfbc2b750 response_code = 200 uri = /metrics/collectors/119eed64-2e24-4d5f-8c42-be6cd8bec8b5/producers?sort_by=id_ascending
61392024-11-28T02:06:09.190ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:55495 req_id = 735849c6-cb8c-41e9-83b8-be94e3342a1a roles = RoleSet { roles: {} } uri = /metrics/collectors/119eed64-2e24-4d5f-8c42-be6cd8bec8b5/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJjNDM0YjQ0OC01MTk5LTQ2NmUtYWYzMi00ZjYyOWUzMWM3YjEifX0%3D
61402024-11-28T02:06:09.190ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:55495 req_id = 735849c6-cb8c-41e9-83b8-be94e3342a1a resource = Database result = Ok(()) uri = /metrics/collectors/119eed64-2e24-4d5f-8c42-be6cd8bec8b5/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJjNDM0YjQ0OC01MTk5LTQ2NmUtYWYzMi00ZjYyOWUzMWM3YjEifX0%3D
61412024-11-28T02:06:09.192ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2110 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:55495 req_id = 735849c6-cb8c-41e9-83b8-be94e3342a1a response_code = 200 uri = /metrics/collectors/119eed64-2e24-4d5f-8c42-be6cd8bec8b5/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJjNDM0YjQ0OC01MTk5LTQ2NmUtYWYzMi00ZjYyOWUzMWM3YjEifX0%3D
61422024-11-28T02:06:09.551ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = bc368aa5-0a3b-44ac-8eda-05ae7b06e715 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
61432024-11-28T02:06:09.551ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = bc368aa5-0a3b-44ac-8eda-05ae7b06e715 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
61442024-11-28T02:06:09.553ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2360 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = bc368aa5-0a3b-44ac-8eda-05ae7b06e715 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
61452024-11-28T02:06:10.478ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 16 reason = Timeout
61462024-11-28T02:06:10.478ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 27 reason = Timeout
61472024-11-28T02:06:10.478ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
61482024-11-28T02:06:10.478ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 27 reason = Timeout
61492024-11-28T02:06:10.478ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 15 reason = Timeout
61502024-11-28T02:06:10.478ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
61512024-11-28T02:06:10.478ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
61522024-11-28T02:06:10.478ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 27 reason = Timeout
61532024-11-28T02:06:10.478ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 16 reason = Timeout
61542024-11-28T02:06:10.478ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
61552024-11-28T02:06:10.478ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 27 reason = Timeout
61562024-11-28T02:06:10.478ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 27 reason = Timeout
61572024-11-28T02:06:10.479ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
61582024-11-28T02:06:10.479ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
61592024-11-28T02:06:10.479ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 15 reason = Timeout
61602024-11-28T02:06:10.479ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 27 reason = Timeout
61612024-11-28T02:06:10.479ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
61622024-11-28T02:06:10.479ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 28 reason = Timeout
61632024-11-28T02:06:10.479ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
61642024-11-28T02:06:10.479ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 14 reason = Timeout
61652024-11-28T02:06:10.479ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
61662024-11-28T02:06:10.479ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 15 reason = Timeout
61672024-11-28T02:06:10.479ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:83 reason = blueprint disabled target_id = 67ac247f-c934-4c78-857a-fba4fbeb9937
61682024-11-28T02:06:10.479ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 15.18\u{b5}s iteration = 15 status = {"enabled":false,"target_id":"67ac247f-c934-4c78-857a-fba4fbeb9937"}
61692024-11-28T02:06:10.479ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 27 reason = Timeout
61702024-11-28T02:06:10.479ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 15 reason = Timeout
61712024-11-28T02:06:10.479ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
61722024-11-28T02:06:10.479ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 27 reason = Timeout
61732024-11-28T02:06:10.479ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
61742024-11-28T02:06:10.479ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
61752024-11-28T02:06:10.479ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
61762024-11-28T02:06:10.479ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
61772024-11-28T02:06:10.479ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 27 reason = Timeout
61782024-11-28T02:06:10.479ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
61792024-11-28T02:06:10.479ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 27 reason = Timeout
61802024-11-28T02:06:10.479ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 27 reason = Timeout
61812024-11-28T02:06:10.479ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 79 reason = Timeout
61822024-11-28T02:06:10.479ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
61832024-11-28T02:06:10.479ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
61842024-11-28T02:06:10.479ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 14 reason = Timeout
61852024-11-28T02:06:10.479ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
61862024-11-28T02:06:10.479ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 29 reason = Timeout
61872024-11-28T02:06:10.479ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
61882024-11-28T02:06:10.479ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 14 reason = Timeout
61892024-11-28T02:06:10.479ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2024-11-28 01:56:10.479123209 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
61902024-11-28T02:06:10.479ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
61912024-11-28T02:06:10.479ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
61922024-11-28T02:06:10.479ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 27 reason = Timeout
61932024-11-28T02:06:10.479ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 79 reason = Timeout
61942024-11-28T02:06:10.479ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
61952024-11-28T02:06:10.479ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 31 reason = Timeout
61962024-11-28T02:06:10.479ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
61972024-11-28T02:06:10.479ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
61982024-11-28T02:06:10.479ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 14 reason = Timeout
61992024-11-28T02:06:10.479ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
62002024-11-28T02:06:10.479ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 14 reason = Timeout
62012024-11-28T02:06:10.479ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:56
62022024-11-28T02:06:10.479ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
62032024-11-28T02:06:10.479ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 15 reason = Timeout
62042024-11-28T02:06:10.479ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
62052024-11-28T02:06:10.479ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
62062024-11-28T02:06:10.479ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 15 reason = Timeout
62072024-11-28T02:06:10.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
62082024-11-28T02:06:10.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
62092024-11-28T02:06:10.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
62102024-11-28T02:06:10.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
62112024-11-28T02:06:10.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
62122024-11-28T02:06:10.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
62132024-11-28T02:06:10.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
62142024-11-28T02:06:10.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
62152024-11-28T02:06:10.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
62162024-11-28T02:06:10.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
62172024-11-28T02:06:10.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
62182024-11-28T02:06:10.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
62192024-11-28T02:06:10.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
62202024-11-28T02:06:10.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
62212024-11-28T02:06:10.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
62222024-11-28T02:06:10.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
62232024-11-28T02:06:10.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
62242024-11-28T02:06:10.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
62252024-11-28T02:06:10.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
62262024-11-28T02:06:10.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
62272024-11-28T02:06:10.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
62282024-11-28T02:06:10.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
62292024-11-28T02:06:10.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
62302024-11-28T02:06:10.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
62312024-11-28T02:06:10.479ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("36d4f615-7461-4364-960c-89ece3aaabaf.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("eb274350-c0b0-4bce-9c8a-502880f2c905.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("eb274350-c0b0-4bce-9c8a-502880f2c905.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::8))) }], valid_until: Instant { tv_sec: 1636, tv_nsec: 949707191 } })
62322024-11-28T02:06:10.479ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("41ea4a99-3038-474e-bf28-2ad338f3243f.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("9402a51b-d30f-4544-93ec-0f5dc7a485e4.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("d23bc5f2-6b92-4992-bdd2-8695e5bf9c0f.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("d23bc5f2-6b92-4992-bdd2-8695e5bf9c0f.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:1::1))) }], valid_until: Instant { tv_sec: 1636, tv_nsec: 949717881 } })
62332024-11-28T02:06:10.480ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } dns_group = external
62342024-11-28T02:06:10.480ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.454415ms iteration = 15 status = {"addresses":["[fd00:1122:3344:101::9]:5353","[fd00:1122:3344:101::8]:5353"]}
62352024-11-28T02:06:10.480ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } dns_group = internal
62362024-11-28T02:06:10.480ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.172921ms iteration = 15 status = {"addresses":["[fd00:1122:3344:3::1]:5353","[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:1::1]:5353"]}
62372024-11-28T02:06:10.481ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
62382024-11-28T02:06:10.481ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 2.817849ms iteration = 27 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
62392024-11-28T02:06:10.482ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
62402024-11-28T02:06:10.483ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
62412024-11-28T02:06:10.483ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
62422024-11-28T02:06:10.483ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
62432024-11-28T02:06:10.485ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 6.756948ms iteration = 14 status = {"errors":[],"found_port_ok":[]}
62442024-11-28T02:06:10.485ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
62452024-11-28T02:06:10.485ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
62462024-11-28T02:06:10.486ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
62472024-11-28T02:06:10.486ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
62482024-11-28T02:06:10.488ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-313847b3-d670-4167-9b09-9e7d3dfcb2a4.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-313847b3-d670-4167-9b09-9e7d3dfcb2a4.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1636, tv_nsec: 958684437 } })
62492024-11-28T02:06:10.489ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1103
62502024-11-28T02:06:10.490ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
62512024-11-28T02:06:10.490ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
62522024-11-28T02:06:10.490ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
62532024-11-28T02:06:10.491ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
62542024-11-28T02:06:10.491ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
62552024-11-28T02:06:10.492ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-313847b3-d670-4167-9b09-9e7d3dfcb2a4.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-313847b3-d670-4167-9b09-9e7d3dfcb2a4.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1636, tv_nsec: 962331070 } })
62562024-11-28T02:06:10.492ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1103
62572024-11-28T02:06:10.493ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2024-11-28 01:56:10.479123209 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
62582024-11-28T02:06:10.493ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 14.418161ms iteration = 14 status = {"expiration":"2024-11-28T01:56:10.479123209Z","pruned":[]}
62592024-11-28T02:06:10.494ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
62602024-11-28T02:06:10.497ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
62612024-11-28T02:06:10.497ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
62622024-11-28T02:06:10.497ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
62632024-11-28T02:06:10.497ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
62642024-11-28T02:06:10.501ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
62652024-11-28T02:06:10.501ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
62662024-11-28T02:06:10.502ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
62672024-11-28T02:06:10.502ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
62682024-11-28T02:06:10.502ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
62692024-11-28T02:06:10.506ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
62702024-11-28T02:06:10.509ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
62712024-11-28T02:06:10.509ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
62722024-11-28T02:06:10.510ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-313847b3-d670-4167-9b09-9e7d3dfcb2a4.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-313847b3-d670-4167-9b09-9e7d3dfcb2a4.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1636, tv_nsec: 980323533 } })
62732024-11-28T02:06:10.510ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1103 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc
62742024-11-28T02:06:10.511ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
62752024-11-28T02:06:10.513ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
62762024-11-28T02:06:10.514ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
62772024-11-28T02:06:10.515ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
62782024-11-28T02:06:10.519ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
62792024-11-28T02:06:10.521ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
62802024-11-28T02:06:10.521ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
62812024-11-28T02:06:10.521ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
62822024-11-28T02:06:10.521ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
62832024-11-28T02:06:10.521ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
62842024-11-28T02:06:10.522ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
62852024-11-28T02:06:10.523ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
62862024-11-28T02:06:10.523ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
62872024-11-28T02:06:10.525ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 46.179482ms iteration = 27 status = {"errors":[],"records_set_to_done":[]}
62882024-11-28T02:06:10.525ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
62892024-11-28T02:06:10.525ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
62902024-11-28T02:06:10.525ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
62912024-11-28T02:06:10.527ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
62922024-11-28T02:06:10.527ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
62932024-11-28T02:06:10.527ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
62942024-11-28T02:06:10.529ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
62952024-11-28T02:06:10.529ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
62962024-11-28T02:06:10.529ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
62972024-11-28T02:06:10.529ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
62982024-11-28T02:06:10.530ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
62992024-11-28T02:06:10.530ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
63002024-11-28T02:06:10.531ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 52.244168ms iteration = 27 status = {"errors":[],"garbage_collect_requested":[]}
63012024-11-28T02:06:10.536ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
63022024-11-28T02:06:10.536ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
63032024-11-28T02:06:10.537ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
63042024-11-28T02:06:10.538ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
63052024-11-28T02:06:10.538ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 59.384522ms iteration = 31 status = {"physical_disks_added":0}
63062024-11-28T02:06:10.539ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 60.616142ms iteration = 27 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[]}
63072024-11-28T02:06:10.540ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
63082024-11-28T02:06:10.540ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
63092024-11-28T02:06:10.542ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
63102024-11-28T02:06:10.542ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:171 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
63112024-11-28T02:06:10.542ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 62.953883ms iteration = 14 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
63122024-11-28T02:06:10.542ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
63132024-11-28T02:06:10.542ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 63.589345ms iteration = 27 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
63142024-11-28T02:06:10.543ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
63152024-11-28T02:06:10.544ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
63162024-11-28T02:06:10.544ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
63172024-11-28T02:06:10.544ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 65.616829ms iteration = 27 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
63182024-11-28T02:06:10.545ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
63192024-11-28T02:06:10.545ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
63202024-11-28T02:06:10.545ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
63212024-11-28T02:06:10.546ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
63222024-11-28T02:06:10.547ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
63232024-11-28T02:06:10.547ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
63242024-11-28T02:06:10.548ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
63252024-11-28T02:06:10.548ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
63262024-11-28T02:06:10.548ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
63272024-11-28T02:06:10.548ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
63282024-11-28T02:06:10.548ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
63292024-11-28T02:06:10.549ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
63302024-11-28T02:06:10.549ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
63312024-11-28T02:06:10.549ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
63322024-11-28T02:06:10.549ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
63332024-11-28T02:06:10.549ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
63342024-11-28T02:06:10.550ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
63352024-11-28T02:06:10.550ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
63362024-11-28T02:06:10.551ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
63372024-11-28T02:06:10.551ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
63382024-11-28T02:06:10.551ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
63392024-11-28T02:06:10.551ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
63402024-11-28T02:06:10.551ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
63412024-11-28T02:06:10.551ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
63422024-11-28T02:06:10.552ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
63432024-11-28T02:06:10.553ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 67ac247f-c934-4c78-857a-fba4fbeb9937, lookup_type: ById(67ac247f-c934-4c78-857a-fba4fbeb9937) } result = Ok(())
63442024-11-28T02:06:10.553ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
63452024-11-28T02:06:10.553ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
63462024-11-28T02:06:10.555ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
63472024-11-28T02:06:10.555ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
63482024-11-28T02:06:10.555ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
63492024-11-28T02:06:10.555ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
63502024-11-28T02:06:10.555ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
63512024-11-28T02:06:10.555ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
63522024-11-28T02:06:10.557ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
63532024-11-28T02:06:10.558ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
63542024-11-28T02:06:10.559ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
63552024-11-28T02:06:10.559ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
63562024-11-28T02:06:10.559ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
63572024-11-28T02:06:10.566ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
63582024-11-28T02:06:10.566ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
63592024-11-28T02:06:10.567ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
63602024-11-28T02:06:10.568ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
63612024-11-28T02:06:10.571ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 313847b3-d670-4167-9b09-9e7d3dfcb2a4, lookup_type: ById(313847b3-d670-4167-9b09-9e7d3dfcb2a4) } result = Ok(())
63622024-11-28T02:06:10.571ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
63632024-11-28T02:06:10.571ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
63642024-11-28T02:06:10.571ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
63652024-11-28T02:06:10.571ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
63662024-11-28T02:06:10.571ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
63672024-11-28T02:06:10.572ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 88362c91-f3c9-4e65-82d3-6897173c49af roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
63682024-11-28T02:06:10.572ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 88362c91-f3c9-4e65-82d3-6897173c49af resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
63692024-11-28T02:06:10.573ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 94.799546ms iteration = 27 status = 0
63702024-11-28T02:06:10.574ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2241 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 88362c91-f3c9-4e65-82d3-6897173c49af response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
63712024-11-28T02:06:10.575ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
63722024-11-28T02:06:10.575ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
63732024-11-28T02:06:10.575ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 96.286913ms iteration = 27 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
63742024-11-28T02:06:10.576ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 67ac247f-c934-4c78-857a-fba4fbeb9937 original_time_created = 2024-11-28 01:53:03.159475 UTC target_id = 67ac247f-c934-4c78-857a-fba4fbeb9937 time_created = 2024-11-28 01:53:03.159475 UTC
63752024-11-28T02:06:10.576ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 97.585315ms iteration = 79 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"67ac247f-c934-4c78-857a-fba4fbeb9937","time_created":"2024-11-28T01:53:03.159475Z"}
63762024-11-28T02:06:10.580ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 100.951264ms iteration = 27 status = {"errors":[],"requests_created_ok":[],"start_invoked_ok":[]}
63772024-11-28T02:06:10.589ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
63782024-11-28T02:06:10.590ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
63792024-11-28T02:06:10.592ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 112.929642ms iteration = 79 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
63802024-11-28T02:06:10.604ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
63812024-11-28T02:06:10.604ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
63822024-11-28T02:06:10.604ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:140 found = 0 not_ready_to_be_deleted = 0
63832024-11-28T02:06:10.604ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 125.074662ms iteration = 14 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
63842024-11-28T02:06:10.607ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
63852024-11-28T02:06:10.611ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
63862024-11-28T02:06:10.611ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
63872024-11-28T02:06:10.611ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
63882024-11-28T02:06:10.615ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
63892024-11-28T02:06:10.615ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
63902024-11-28T02:06:10.615ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
63912024-11-28T02:06:10.615ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = a7e6af61-408c-483d-b043-c27ccf55708e dns_zone_name = oxide.test found_so_far = 0 version = 2
63922024-11-28T02:06:10.615ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
63932024-11-28T02:06:10.616ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
63942024-11-28T02:06:10.617ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
63952024-11-28T02:06:10.617ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
63962024-11-28T02:06:10.618ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "40f82e50-05e7-4cfa-b728-9504e7bef69f", "content-length": "26", "date": "Thu, 28 Nov 2024 02:06:10 GMT"} })
63972024-11-28T02:06:10.618ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
63982024-11-28T02:06:10.619ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
63992024-11-28T02:06:10.620ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
64002024-11-28T02:06:10.621ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2024-11-28T01:53:08.890632Z, time_modified: 2024-11-28T01:53:08.890632Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 7e159834-997a-4c03-b9db-1e91d783221c, db_silo: Silo { identity: SiloIdentity { id: 7e159834-997a-4c03-b9db-1e91d783221c, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-11-28T01:53:43.308696Z, time_modified: 2024-11-28T01:53:43.308696Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "bc39d1b3b4bb84a49d6458ed7aedc5bca9e77742c7b35882b64d188c0602d306" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 7e159834-997a-4c03-b9db-1e91d783221c, db_silo: Silo { identity: SiloIdentity { id: 7e159834-997a-4c03-b9db-1e91d783221c, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-11-28T01:53:43.308696Z, time_modified: 2024-11-28T01:53:43.308696Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "bc39d1b3b4bb84a49d6458ed7aedc5bca9e77742c7b35882b64d188c0602d306" }] }) }
64012024-11-28T02:06:10.621ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 142.547916ms iteration = 15 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"7e159834-997a-4c03-b9db-1e91d783221c","tls_certs":["bc39d1b3b4bb84a49d6458ed7aedc5bca9e77742c7b35882b64d188c0602d306"]}},"default_endpoint":{"silo_id":"7e159834-997a-4c03-b9db-1e91d783221c","tls_certs":["bc39d1b3b4bb84a49d6458ed7aedc5bca9e77742c7b35882b64d188c0602d306"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
64022024-11-28T02:06:10.624ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
64032024-11-28T02:06:10.625ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
64042024-11-28T02:06:10.636ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
64052024-11-28T02:06:10.636ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
64062024-11-28T02:06:10.636ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "01c4d313-4e41-4079-aad7-89df4594a2df", "content-length": "26", "date": "Thu, 28 Nov 2024 02:06:10 GMT"} })
64072024-11-28T02:06:10.636ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
64082024-11-28T02:06:10.638ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
64092024-11-28T02:06:10.644ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
64102024-11-28T02:06:10.644ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 808a324e-c913-4bfc-91a5-d43c961cd511 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
64112024-11-28T02:06:10.644ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
64122024-11-28T02:06:10.645ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
64132024-11-28T02:06:10.645ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
64142024-11-28T02:06:10.648ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
64152024-11-28T02:06:10.648ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
64162024-11-28T02:06:10.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
64172024-11-28T02:06:10.650ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
64182024-11-28T02:06:10.654ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
64192024-11-28T02:06:10.654ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
64202024-11-28T02:06:10.654ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
64212024-11-28T02:06:10.660ZWARNnexus (BackgroundTasks): instance reincarnation completed with errors background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:103 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0 query_errors = 0 restart_errors = 0
64222024-11-28T02:06:10.660ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 180.869422ms iteration = 14 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
64232024-11-28T02:06:10.660ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
64242024-11-28T02:06:10.662ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
64252024-11-28T02:06:10.662ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
64262024-11-28T02:06:10.662ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
64272024-11-28T02:06:10.704ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1111 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc zone_address = fd00:1122:3344:101::2
64282024-11-28T02:06:10.704ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
64292024-11-28T02:06:10.705ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "9f0dfca8-ab50-4d7f-a3a1-989a5e9f6a91", "content-length": "26", "date": "Thu, 28 Nov 2024 02:06:10 GMT"} })
64302024-11-28T02:06:10.705ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1116 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
64312024-11-28T02:06:10.711ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc
64322024-11-28T02:06:10.794ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
64332024-11-28T02:06:10.806ZINFOnexus (BackgroundTasks): VPC route manager sled 313847b3-d670-4167-9b09-9e7d3dfcb2a4 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
64342024-11-28T02:06:10.806ZDEBGnexus (BackgroundTasks): client request SledAgent = 313847b3-d670-4167-9b09-9e7d3dfcb2a4 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
64352024-11-28T02:06:10.807ZDEBGnexus (BackgroundTasks): client response SledAgent = 313847b3-d670-4167-9b09-9e7d3dfcb2a4 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "d5989f86-b8a3-48b7-a33d-c85b1e36757b", "content-length": "372", "date": "Thu, 28 Nov 2024 02:06:10 GMT"} })
64362024-11-28T02:06:10.807ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
64372024-11-28T02:06:10.807ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
64382024-11-28T02:06:10.809ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
64392024-11-28T02:06:10.809ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "e3575271-245a-4a9d-bd05-8f3826270f94", "content-length": "2", "date": "Thu, 28 Nov 2024 02:06:09 GMT"} })
64402024-11-28T02:06:10.810ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = a7e6af61-408c-483d-b043-c27ccf55708e dns_zone_name = oxide.test found_so_far = 1 version = 2
64412024-11-28T02:06:10.810ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
64422024-11-28T02:06:10.810ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2024-11-28 01:53:43.351579 UTC dns_group = external generation = 2
64432024-11-28T02:06:10.810ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 331.706721ms iteration = 15 status = {"generation":2}
64442024-11-28T02:06:10.811ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
64452024-11-28T02:06:10.811ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
64462024-11-28T02:06:10.811ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 808a324e-c913-4bfc-91a5-d43c961cd511 dns_zone_name = control-plane.oxide.internal found_so_far = 51 version = 1
64472024-11-28T02:06:10.811ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
64482024-11-28T02:06:10.811ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2024-11-28 01:53:42.502562 UTC dns_group = internal generation = 1
64492024-11-28T02:06:10.811ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 332.550616ms iteration = 15 status = {"generation":1}
64502024-11-28T02:06:10.811ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
64512024-11-28T02:06:10.813ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
64522024-11-28T02:06:10.813ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
64532024-11-28T02:06:10.814ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 335.130251ms iteration = 27 status = null
64542024-11-28T02:06:10.814ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
64552024-11-28T02:06:10.821ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
64562024-11-28T02:06:10.822ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "91e7852b-f680-4ae3-a830-b0cf5c83bcab", "date": "Thu, 28 Nov 2024 02:06:10 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
64572024-11-28T02:06:10.822ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
64582024-11-28T02:06:10.823ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "5ea96f02-c7c8-4990-a30f-4f548752393a", "date": "Thu, 28 Nov 2024 02:06:10 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
64592024-11-28T02:06:10.823ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
64602024-11-28T02:06:10.826ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
64612024-11-28T02:06:10.826ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 347.761081ms iteration = 16 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
64622024-11-28T02:06:10.830ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 351.77577ms iteration = 27 status = {}
64632024-11-28T02:06:10.874ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
64642024-11-28T02:06:10.874ZDEBGnexus (BackgroundTasks): client request SledAgent = 313847b3-d670-4167-9b09-9e7d3dfcb2a4 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
64652024-11-28T02:06:10.875ZDEBGnexus (BackgroundTasks): client response SledAgent = 313847b3-d670-4167-9b09-9e7d3dfcb2a4 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "8d7b563d-24e6-49ab-a002-96b4dda914b5", "date": "Thu, 28 Nov 2024 02:06:10 GMT"} })
64662024-11-28T02:06:10.875ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
64672024-11-28T02:06:10.875ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
64682024-11-28T02:06:10.877ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
64692024-11-28T02:06:10.877ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
64702024-11-28T02:06:10.880ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
64712024-11-28T02:06:10.880ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
64722024-11-28T02:06:10.884ZINFOnexus (BackgroundTasks): VPC route manager sled 313847b3-d670-4167-9b09-9e7d3dfcb2a4 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
64732024-11-28T02:06:10.884ZDEBGnexus (BackgroundTasks): client request SledAgent = 313847b3-d670-4167-9b09-9e7d3dfcb2a4 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
64742024-11-28T02:06:10.884ZDEBGnexus (BackgroundTasks): client response SledAgent = 313847b3-d670-4167-9b09-9e7d3dfcb2a4 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "798a6624-a24a-4994-8eea-c7766f25a5fe", "date": "Thu, 28 Nov 2024 02:06:10 GMT"} })
64752024-11-28T02:06:10.886ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 407.804493ms iteration = 29 status = {}
64762024-11-28T02:06:10.890ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
64772024-11-28T02:06:10.891ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "43ddb8d0-6320-4bbb-95c2-fc55c435a80e", "date": "Thu, 28 Nov 2024 02:06:10 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
64782024-11-28T02:06:10.891ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
64792024-11-28T02:06:10.893ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "4d8194ff-b2f9-4070-9f58-17c93dd6403a", "date": "Thu, 28 Nov 2024 02:06:10 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
64802024-11-28T02:06:10.893ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
64812024-11-28T02:06:10.895ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "c1c4df0f-65ff-4cab-9615-0ba37d19e14f", "date": "Thu, 28 Nov 2024 02:06:10 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
64822024-11-28T02:06:10.895ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
64832024-11-28T02:06:10.897ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
64842024-11-28T02:06:10.897ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 418.391207ms iteration = 16 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
64852024-11-28T02:06:10.938ZDEBGnexus (BackgroundTasks): client request SledAgent = 313847b3-d670-4167-9b09-9e7d3dfcb2a4 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
64862024-11-28T02:06:10.939ZDEBGnexus (BackgroundTasks): client response SledAgent = 313847b3-d670-4167-9b09-9e7d3dfcb2a4 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "bdea8781-3104-40ed-8ec5-44339c05c12a", "content-length": "691", "date": "Thu, 28 Nov 2024 02:06:10 GMT"} })
64872024-11-28T02:06:10.939ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
64882024-11-28T02:06:10.939ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
64892024-11-28T02:06:10.939ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
64902024-11-28T02:06:10.939ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 460.905086ms iteration = 28 status = {}
64912024-11-28T02:06:10.943ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
64922024-11-28T02:06:10.944ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
64932024-11-28T02:06:10.945ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 4bb2986f-2e92-4ac3-9dc1-591b6bb60b3e, rack_id: 92a8ff6e-21d3-4c28-bcad-e323847985dc, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(ea300318-9d62-42ed-80ea-21c86590a166) } rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc switch_location = Switch0
64942024-11-28T02:06:10.945ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
64952024-11-28T02:06:10.945ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
64962024-11-28T02:06:10.956ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
64972024-11-28T02:06:10.956ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "46e9a1b2-0a5e-47d2-a86e-25c31b6f65d1", "content-length": "134", "date": "Thu, 28 Nov 2024 02:06:10 GMT"} })
64982024-11-28T02:06:10.956ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1746 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
64992024-11-28T02:06:10.956ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1773 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = ea300318-9d62-42ed-80ea-21c86590a166
65002024-11-28T02:06:10.956ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
65012024-11-28T02:06:10.956ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
65022024-11-28T02:06:10.956ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
65032024-11-28T02:06:10.956ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
65042024-11-28T02:06:10.956ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
65052024-11-28T02:06:10.956ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1920 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
65062024-11-28T02:06:10.956ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
65072024-11-28T02:06:10.956ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
65082024-11-28T02:06:10.957ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc
65092024-11-28T02:06:10.957ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
65102024-11-28T02:06:10.957ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
65112024-11-28T02:06:10.958ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
65122024-11-28T02:06:10.958ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "b52274e0-87d3-4bb7-b3b8-37afcd6d5cb7", "content-length": "2", "date": "Thu, 28 Nov 2024 02:06:10 GMT"} })
65132024-11-28T02:06:10.958ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
65142024-11-28T02:06:10.958ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "b4672248-691a-42d3-afad-939e5ba9442b", "content-length": "52", "date": "Thu, 28 Nov 2024 02:06:10 GMT"} })
65152024-11-28T02:06:10.958ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc switch_location = Switch0
65162024-11-28T02:06:10.958ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
65172024-11-28T02:06:11.056ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "ea456720-096a-4e05-a66f-21d9bd00baef", "date": "Thu, 28 Nov 2024 02:06:10 GMT"} })
65182024-11-28T02:06:11.056ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc switch = Switch0
65192024-11-28T02:06:11.056ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
65202024-11-28T02:06:11.057ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "862c2b28-4c10-4ad3-94e8-e32e19e61557", "content-length": "682", "date": "Thu, 28 Nov 2024 02:06:11 GMT"} })
65212024-11-28T02:06:11.057ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
65222024-11-28T02:06:11.057ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
65232024-11-28T02:06:11.060ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
65242024-11-28T02:06:11.060ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
65252024-11-28T02:06:11.063ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
65262024-11-28T02:06:11.063ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
65272024-11-28T02:06:11.065ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
65282024-11-28T02:06:11.066ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
65292024-11-28T02:06:11.074ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 595.29381ms iteration = 27 status = {}
65302024-11-28T02:06:11.595ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 7f211c7d-3aef-4325-9d6c-c0d0090260e8 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
65312024-11-28T02:06:11.596ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 7f211c7d-3aef-4325-9d6c-c0d0090260e8 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
65322024-11-28T02:06:11.598ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2249 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 7f211c7d-3aef-4325-9d6c-c0d0090260e8 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
65332024-11-28T02:06:12.616ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 15383bef-8d40-4632-bc7f-b5523bbe47ac roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
65342024-11-28T02:06:12.617ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 15383bef-8d40-4632-bc7f-b5523bbe47ac resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
65352024-11-28T02:06:12.619ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2341 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 15383bef-8d40-4632-bc7f-b5523bbe47ac response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
65362024-11-28T02:06:13.638ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 727e49a2-5e5d-4894-89df-d447efb3be90 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
65372024-11-28T02:06:13.639ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 727e49a2-5e5d-4894-89df-d447efb3be90 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
65382024-11-28T02:06:13.641ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 3022 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 727e49a2-5e5d-4894-89df-d447efb3be90 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
65392024-11-28T02:06:14.195ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1389 local_addr = [fd00:1122:3344:101::b]:55621 method = GET remote_addr = [fd00:1122:3344:101::d]:56417 req_id = 3d51be21-a15c-4ed6-b09c-fa8144a77b50 response_code = 200 uri = /2469a748-bcd2-4e0d-99cc-f79def0452cc
65402024-11-28T02:06:14.661ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 8ab70eb0-ea1c-48ed-a35f-4c205574c2ed roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
65412024-11-28T02:06:14.661ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 8ab70eb0-ea1c-48ed-a35f-4c205574c2ed resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
65422024-11-28T02:06:14.663ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2322 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 8ab70eb0-ea1c-48ed-a35f-4c205574c2ed response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
65432024-11-28T02:06:15.321ZDEBGnexus (producer-registration-task): registering / renewing oximeter producer lease with Nexus
65442024-11-28T02:06:15.321ZDEBGnexus (producer-registration-task): using nexus address for registration addr = [fd00:1122:3344:101::b]:12221
65452024-11-28T02:06:15.351ZDEBGnexus (producer-registration-task): client request body = Some(Body) method = POST uri = http://[fd00:1122:3344:101::b]:12221/metrics/producers
65462024-11-28T02:06:15.352ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::b]:12221 remote_addr = [fd00:1122:3344:101::b]:45495
65472024-11-28T02:06:15.352ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::b]:45495 req_id = 75eb6641-baa1-448c-9e57-7a06550d4b6c roles = RoleSet { roles: {} } uri = /metrics/producers
65482024-11-28T02:06:15.352ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::b]:45495 req_id = 75eb6641-baa1-448c-9e57-7a06550d4b6c resource = Database result = Ok(()) uri = /metrics/producers
65492024-11-28T02:06:15.387ZINFOnexus: assigned collector to new producer collector_id = 119eed64-2e24-4d5f-8c42-be6cd8bec8b5 file = nexus/src/app/oximeter.rs:76 producer_id = 2469a748-bcd2-4e0d-99cc-f79def0452cc
65502024-11-28T02:06:15.387ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 35810 local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::b]:45495 req_id = 75eb6641-baa1-448c-9e57-7a06550d4b6c response_code = 201 uri = /metrics/producers
65512024-11-28T02:06:15.388ZDEBGnexus (producer-registration-task): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::b]:12221/metrics/producers", status: 201, headers: {"content-type": "application/json", "x-request-id": "75eb6641-baa1-448c-9e57-7a06550d4b6c", "content-length": "41", "date": "Thu, 28 Nov 2024 02:06:15 GMT"} })
65522024-11-28T02:06:15.389ZDEBGnexus (producer-registration-task): registered with nexus successfully
65532024-11-28T02:06:15.389ZDEBGnexus (producer-registration-task): pausing until time to renew lease lease_duration = 600s wait_period = 150s
65542024-11-28T02:06:15.682ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 5acd212f-3df6-4fdf-8792-bbc7b51a06cd roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
65552024-11-28T02:06:15.682ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 5acd212f-3df6-4fdf-8792-bbc7b51a06cd resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
65562024-11-28T02:06:15.684ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2775 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 5acd212f-3df6-4fdf-8792-bbc7b51a06cd response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
65572024-11-28T02:06:16.703ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 3ec14a90-c206-4539-b5eb-f90ba5120f5d roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
65582024-11-28T02:06:16.703ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 3ec14a90-c206-4539-b5eb-f90ba5120f5d resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
65592024-11-28T02:06:16.706ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2967 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 3ec14a90-c206-4539-b5eb-f90ba5120f5d response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
65602024-11-28T02:06:17.725ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 87bb8e59-cb15-4945-9bf4-cf5cd7465d05 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
65612024-11-28T02:06:17.725ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 87bb8e59-cb15-4945-9bf4-cf5cd7465d05 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
65622024-11-28T02:06:17.727ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2216 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 87bb8e59-cb15-4945-9bf4-cf5cd7465d05 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
65632024-11-28T02:06:18.748ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = eda34495-e612-42d9-9c1d-a1a3793b02eb roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
65642024-11-28T02:06:18.748ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = eda34495-e612-42d9-9c1d-a1a3793b02eb resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
65652024-11-28T02:06:18.750ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2456 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = eda34495-e612-42d9-9c1d-a1a3793b02eb response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
65662024-11-28T02:06:19.769ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 1d4e604a-81e9-46e9-b05c-690ff710fb66 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
65672024-11-28T02:06:19.769ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 1d4e604a-81e9-46e9-b05c-690ff710fb66 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
65682024-11-28T02:06:19.771ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1740 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 1d4e604a-81e9-46e9-b05c-690ff710fb66 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
65692024-11-28T02:06:20.479ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 80 reason = Timeout
65702024-11-28T02:06:20.479ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
65712024-11-28T02:06:20.479ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 80 reason = Timeout
65722024-11-28T02:06:20.479ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
65732024-11-28T02:06:20.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
65742024-11-28T02:06:20.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
65752024-11-28T02:06:20.481ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
65762024-11-28T02:06:20.481ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
65772024-11-28T02:06:20.481ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
65782024-11-28T02:06:20.484ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
65792024-11-28T02:06:20.484ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
65802024-11-28T02:06:20.484ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
65812024-11-28T02:06:20.486ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
65822024-11-28T02:06:20.486ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
65832024-11-28T02:06:20.487ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
65842024-11-28T02:06:20.490ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 67ac247f-c934-4c78-857a-fba4fbeb9937, lookup_type: ById(67ac247f-c934-4c78-857a-fba4fbeb9937) } result = Ok(())
65852024-11-28T02:06:20.490ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
65862024-11-28T02:06:20.490ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
65872024-11-28T02:06:20.500ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 67ac247f-c934-4c78-857a-fba4fbeb9937 original_time_created = 2024-11-28 01:53:03.159475 UTC target_id = 67ac247f-c934-4c78-857a-fba4fbeb9937 time_created = 2024-11-28 01:53:03.159475 UTC
65882024-11-28T02:06:20.500ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 21.373111ms iteration = 80 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"67ac247f-c934-4c78-857a-fba4fbeb9937","time_created":"2024-11-28T01:53:03.159475Z"}
65892024-11-28T02:06:20.509ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 30.181095ms iteration = 80 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
65902024-11-28T02:06:20.789ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 329540e0-1883-4670-a762-5796a90e837c roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
65912024-11-28T02:06:20.790ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 329540e0-1883-4670-a762-5796a90e837c resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
65922024-11-28T02:06:20.791ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2252 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 329540e0-1883-4670-a762-5796a90e837c response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
65932024-11-28T02:06:21.811ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 2beaa0ec-dc89-46ef-89b0-a17552e75e3e roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
65942024-11-28T02:06:21.811ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 2beaa0ec-dc89-46ef-89b0-a17552e75e3e resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
65952024-11-28T02:06:21.813ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2127 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 2beaa0ec-dc89-46ef-89b0-a17552e75e3e response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
65962024-11-28T02:06:22.832ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 983fcefb-c4e7-43c0-9273-cd25f586154b roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
65972024-11-28T02:06:22.832ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 983fcefb-c4e7-43c0-9273-cd25f586154b resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
65982024-11-28T02:06:22.834ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2676 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 983fcefb-c4e7-43c0-9273-cd25f586154b response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
65992024-11-28T02:06:23.853ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 7652dae2-bd72-4963-9b25-4113d5acd633 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
66002024-11-28T02:06:23.853ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 7652dae2-bd72-4963-9b25-4113d5acd633 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
66012024-11-28T02:06:23.855ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2023 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 7652dae2-bd72-4963-9b25-4113d5acd633 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
66022024-11-28T02:06:24.186ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:55495 req_id = 4f7053e9-bd65-42d3-b957-8316d8bd9597 roles = RoleSet { roles: {} } uri = /metrics/collectors/119eed64-2e24-4d5f-8c42-be6cd8bec8b5/producers?sort_by=id_ascending
66032024-11-28T02:06:24.186ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:55495 req_id = 4f7053e9-bd65-42d3-b957-8316d8bd9597 resource = Database result = Ok(()) uri = /metrics/collectors/119eed64-2e24-4d5f-8c42-be6cd8bec8b5/producers?sort_by=id_ascending
66042024-11-28T02:06:24.189ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 3316 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:55495 req_id = 4f7053e9-bd65-42d3-b957-8316d8bd9597 response_code = 200 uri = /metrics/collectors/119eed64-2e24-4d5f-8c42-be6cd8bec8b5/producers?sort_by=id_ascending
66052024-11-28T02:06:24.189ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:55495 req_id = ee242147-14ae-46a6-ace4-ee47f7111c9c roles = RoleSet { roles: {} } uri = /metrics/collectors/119eed64-2e24-4d5f-8c42-be6cd8bec8b5/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJjNDM0YjQ0OC01MTk5LTQ2NmUtYWYzMi00ZjYyOWUzMWM3YjEifX0%3D
66062024-11-28T02:06:24.190ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:55495 req_id = ee242147-14ae-46a6-ace4-ee47f7111c9c resource = Database result = Ok(()) uri = /metrics/collectors/119eed64-2e24-4d5f-8c42-be6cd8bec8b5/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJjNDM0YjQ0OC01MTk5LTQ2NmUtYWYzMi00ZjYyOWUzMWM3YjEifX0%3D
66072024-11-28T02:06:24.192ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2456 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:55495 req_id = ee242147-14ae-46a6-ace4-ee47f7111c9c response_code = 200 uri = /metrics/collectors/119eed64-2e24-4d5f-8c42-be6cd8bec8b5/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJjNDM0YjQ0OC01MTk5LTQ2NmUtYWYzMi00ZjYyOWUzMWM3YjEifX0%3D
66082024-11-28T02:06:24.194ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1253 local_addr = [fd00:1122:3344:101::b]:55621 method = GET remote_addr = [fd00:1122:3344:101::d]:56417 req_id = 1f2a3200-54a4-4f26-a32e-b317e4d271ad response_code = 200 uri = /2469a748-bcd2-4e0d-99cc-f79def0452cc
66092024-11-28T02:06:24.874ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 14459d1e-190b-4821-b524-45825f76734c roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
66102024-11-28T02:06:24.875ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 14459d1e-190b-4821-b524-45825f76734c resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
66112024-11-28T02:06:24.877ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2591 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 14459d1e-190b-4821-b524-45825f76734c response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
66122024-11-28T02:06:25.896ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 9accda9f-d741-4d3a-8571-0a9e43490110 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
66132024-11-28T02:06:25.897ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 9accda9f-d741-4d3a-8571-0a9e43490110 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
66142024-11-28T02:06:25.899ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2343 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 9accda9f-d741-4d3a-8571-0a9e43490110 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
66152024-11-28T02:06:26.917ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 169de53f-90fe-4705-957c-fedc909a2492 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
66162024-11-28T02:06:26.917ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 169de53f-90fe-4705-957c-fedc909a2492 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
66172024-11-28T02:06:26.919ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2104 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 169de53f-90fe-4705-957c-fedc909a2492 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
66182024-11-28T02:06:27.938ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 735f65fe-ad47-4482-8187-bedace69ac81 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
66192024-11-28T02:06:27.939ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 735f65fe-ad47-4482-8187-bedace69ac81 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
66202024-11-28T02:06:27.941ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2367 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 735f65fe-ad47-4482-8187-bedace69ac81 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
66212024-11-28T02:06:28.959ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = a2661b1c-8908-404f-8142-75138dc92f97 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
66222024-11-28T02:06:28.960ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = a2661b1c-8908-404f-8142-75138dc92f97 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
66232024-11-28T02:06:28.962ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2646 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = a2661b1c-8908-404f-8142-75138dc92f97 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
66242024-11-28T02:06:29.982ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 8f80eb83-06b1-4f3c-bb01-f5558991515d roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
66252024-11-28T02:06:29.982ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 8f80eb83-06b1-4f3c-bb01-f5558991515d resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
66262024-11-28T02:06:29.984ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2269 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 8f80eb83-06b1-4f3c-bb01-f5558991515d response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
66272024-11-28T02:06:30.479ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 81 reason = Timeout
66282024-11-28T02:06:30.479ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 81 reason = Timeout
66292024-11-28T02:06:30.479ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
66302024-11-28T02:06:30.479ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
66312024-11-28T02:06:30.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
66322024-11-28T02:06:30.480ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
66332024-11-28T02:06:30.511ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
66342024-11-28T02:06:30.511ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
66352024-11-28T02:06:30.513ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 33.555163ms iteration = 81 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
66362024-11-28T02:06:30.514ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
66372024-11-28T02:06:30.517ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
66382024-11-28T02:06:30.517ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
66392024-11-28T02:06:30.517ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
66402024-11-28T02:06:30.519ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
66412024-11-28T02:06:30.520ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
66422024-11-28T02:06:30.548ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
66432024-11-28T02:06:30.551ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 67ac247f-c934-4c78-857a-fba4fbeb9937, lookup_type: ById(67ac247f-c934-4c78-857a-fba4fbeb9937) } result = Ok(())
66442024-11-28T02:06:30.551ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
66452024-11-28T02:06:30.551ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
66462024-11-28T02:06:30.562ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 67ac247f-c934-4c78-857a-fba4fbeb9937 original_time_created = 2024-11-28 01:53:03.159475 UTC target_id = 67ac247f-c934-4c78-857a-fba4fbeb9937 time_created = 2024-11-28 01:53:03.159475 UTC
66472024-11-28T02:06:30.562ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 83.149534ms iteration = 81 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"67ac247f-c934-4c78-857a-fba4fbeb9937","time_created":"2024-11-28T01:53:03.159475Z"}
66482024-11-28T02:06:31.002ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = fde3e22e-a6d9-4e03-a0bc-f5af4e9325aa roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
66492024-11-28T02:06:31.003ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = fde3e22e-a6d9-4e03-a0bc-f5af4e9325aa resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
66502024-11-28T02:06:31.004ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1669 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = fde3e22e-a6d9-4e03-a0bc-f5af4e9325aa response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
66512024-11-28T02:06:32.023ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 40e48985-8808-4093-8ea4-714ccee8b6b1 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
66522024-11-28T02:06:32.023ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 40e48985-8808-4093-8ea4-714ccee8b6b1 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
66532024-11-28T02:06:32.026ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2929 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 40e48985-8808-4093-8ea4-714ccee8b6b1 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
66542024-11-28T02:06:33.046ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 8e3e8098-c29a-42f9-9f9d-4a1de374da08 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
66552024-11-28T02:06:33.046ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 8e3e8098-c29a-42f9-9f9d-4a1de374da08 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
66562024-11-28T02:06:33.048ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2635 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 8e3e8098-c29a-42f9-9f9d-4a1de374da08 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
66572024-11-28T02:06:34.067ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 456ed9bd-18ec-4cde-893d-539eda01eefc roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
66582024-11-28T02:06:34.067ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 456ed9bd-18ec-4cde-893d-539eda01eefc resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
66592024-11-28T02:06:34.070ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2558 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 456ed9bd-18ec-4cde-893d-539eda01eefc response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
66602024-11-28T02:06:34.195ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1477 local_addr = [fd00:1122:3344:101::b]:55621 method = GET remote_addr = [fd00:1122:3344:101::d]:56417 req_id = 4f25857e-5b36-46b8-8655-6884855cdb88 response_code = 200 uri = /2469a748-bcd2-4e0d-99cc-f79def0452cc
66612024-11-28T02:06:35.088ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 4615e0ea-fd2f-4195-a62a-4343c6f1a1bc roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
66622024-11-28T02:06:35.088ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 4615e0ea-fd2f-4195-a62a-4343c6f1a1bc resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
66632024-11-28T02:06:35.090ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2562 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 4615e0ea-fd2f-4195-a62a-4343c6f1a1bc response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
66642024-11-28T02:06:36.110ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = e2d983d0-e15a-4700-8ddc-85db09cbaa14 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
66652024-11-28T02:06:36.110ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = e2d983d0-e15a-4700-8ddc-85db09cbaa14 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
66662024-11-28T02:06:36.112ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2057 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = e2d983d0-e15a-4700-8ddc-85db09cbaa14 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
66672024-11-28T02:06:37.130ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = e81b84da-455d-4a53-971f-25f7532efaa8 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
66682024-11-28T02:06:37.130ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = e81b84da-455d-4a53-971f-25f7532efaa8 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
66692024-11-28T02:06:37.132ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2279 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = e81b84da-455d-4a53-971f-25f7532efaa8 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
66702024-11-28T02:06:38.151ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 434e9708-4581-49a3-85f6-99c4f3ca8705 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
66712024-11-28T02:06:38.151ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 434e9708-4581-49a3-85f6-99c4f3ca8705 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
66722024-11-28T02:06:38.153ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1977 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 434e9708-4581-49a3-85f6-99c4f3ca8705 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
66732024-11-28T02:06:38.609ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:40402 req_id = 610d160a-7b05-42ce-aa80-8315f46207ee roles = RoleSet { roles: {} } uri = /nat/ipv4/changeset/10?limit=100
66742024-11-28T02:06:38.609ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:40402 req_id = 610d160a-7b05-42ce-aa80-8315f46207ee resource = Database result = Ok(()) uri = /nat/ipv4/changeset/10?limit=100
66752024-11-28T02:06:38.613ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 3778 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:40402 req_id = 610d160a-7b05-42ce-aa80-8315f46207ee response_code = 200 uri = /nat/ipv4/changeset/10?limit=100
66762024-11-28T02:06:39.171ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 835e8b72-f3b3-43d6-a2b6-829f26a1c632 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
66772024-11-28T02:06:39.171ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 835e8b72-f3b3-43d6-a2b6-829f26a1c632 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
66782024-11-28T02:06:39.173ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2324 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 835e8b72-f3b3-43d6-a2b6-829f26a1c632 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
66792024-11-28T02:06:39.186ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:55495 req_id = 505a43b8-8560-45e8-a792-f6acca058359 roles = RoleSet { roles: {} } uri = /metrics/collectors/119eed64-2e24-4d5f-8c42-be6cd8bec8b5/producers?sort_by=id_ascending
66802024-11-28T02:06:39.186ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:55495 req_id = 505a43b8-8560-45e8-a792-f6acca058359 resource = Database result = Ok(()) uri = /metrics/collectors/119eed64-2e24-4d5f-8c42-be6cd8bec8b5/producers?sort_by=id_ascending
66812024-11-28T02:06:39.189ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 3462 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:55495 req_id = 505a43b8-8560-45e8-a792-f6acca058359 response_code = 200 uri = /metrics/collectors/119eed64-2e24-4d5f-8c42-be6cd8bec8b5/producers?sort_by=id_ascending
66822024-11-28T02:06:39.190ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:55495 req_id = 1072ae37-0921-460e-935d-42828cbd1a77 roles = RoleSet { roles: {} } uri = /metrics/collectors/119eed64-2e24-4d5f-8c42-be6cd8bec8b5/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJjNDM0YjQ0OC01MTk5LTQ2NmUtYWYzMi00ZjYyOWUzMWM3YjEifX0%3D
66832024-11-28T02:06:39.190ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:55495 req_id = 1072ae37-0921-460e-935d-42828cbd1a77 resource = Database result = Ok(()) uri = /metrics/collectors/119eed64-2e24-4d5f-8c42-be6cd8bec8b5/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJjNDM0YjQ0OC01MTk5LTQ2NmUtYWYzMi00ZjYyOWUzMWM3YjEifX0%3D
66842024-11-28T02:06:39.192ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2433 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:55495 req_id = 1072ae37-0921-460e-935d-42828cbd1a77 response_code = 200 uri = /metrics/collectors/119eed64-2e24-4d5f-8c42-be6cd8bec8b5/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJjNDM0YjQ0OC01MTk5LTQ2NmUtYWYzMi00ZjYyOWUzMWM3YjEifX0%3D
66852024-11-28T02:06:40.192ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = c17ced29-7702-49bd-a4d3-77e2b15df5dc roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
66862024-11-28T02:06:40.192ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = c17ced29-7702-49bd-a4d3-77e2b15df5dc resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
66872024-11-28T02:06:40.194ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2497 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = c17ced29-7702-49bd-a4d3-77e2b15df5dc response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
66882024-11-28T02:06:40.479ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 82 reason = Timeout
66892024-11-28T02:06:40.479ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 28 reason = Timeout
66902024-11-28T02:06:40.479ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
66912024-11-28T02:06:40.479ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
66922024-11-28T02:06:40.479ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 82 reason = Timeout
66932024-11-28T02:06:40.479ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 28 reason = Timeout
66942024-11-28T02:06:40.479ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
66952024-11-28T02:06:40.479ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
66962024-11-28T02:06:40.479ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 28 reason = Timeout
66972024-11-28T02:06:40.479ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
66982024-11-28T02:06:40.479ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 29 reason = Timeout
66992024-11-28T02:06:40.479ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
67002024-11-28T02:06:40.479ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 30 reason = Timeout
67012024-11-28T02:06:40.479ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
67022024-11-28T02:06:40.479ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
67032024-11-28T02:06:40.479ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 28 reason = Timeout
67042024-11-28T02:06:40.479ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 28 reason = Timeout
67052024-11-28T02:06:40.479ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 28 reason = Timeout
67062024-11-28T02:06:40.479ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
67072024-11-28T02:06:40.479ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
67082024-11-28T02:06:40.479ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
67092024-11-28T02:06:40.479ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 28 reason = Timeout
67102024-11-28T02:06:40.479ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 28 reason = Timeout
67112024-11-28T02:06:40.479ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
67122024-11-28T02:06:40.479ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
67132024-11-28T02:06:40.479ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 28 reason = Timeout
67142024-11-28T02:06:40.479ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
67152024-11-28T02:06:40.479ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 32 reason = Timeout
67162024-11-28T02:06:40.479ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
67172024-11-28T02:06:40.479ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 28 reason = Timeout
67182024-11-28T02:06:40.480ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 28 reason = Timeout
67192024-11-28T02:06:40.480ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
67202024-11-28T02:06:40.480ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
67212024-11-28T02:06:40.480ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 28 reason = Timeout
67222024-11-28T02:06:40.480ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
67232024-11-28T02:06:40.480ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
67242024-11-28T02:06:40.480ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
67252024-11-28T02:06:40.480ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
67262024-11-28T02:06:40.480ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
67272024-11-28T02:06:40.480ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
67282024-11-28T02:06:40.480ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
67292024-11-28T02:06:40.480ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
67302024-11-28T02:06:40.480ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
67312024-11-28T02:06:40.480ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
67322024-11-28T02:06:40.480ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
67332024-11-28T02:06:40.480ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
67342024-11-28T02:06:40.480ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
67352024-11-28T02:06:40.480ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
67362024-11-28T02:06:40.480ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
67372024-11-28T02:06:40.480ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
67382024-11-28T02:06:40.480ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
67392024-11-28T02:06:40.484ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
67402024-11-28T02:06:40.485ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
67412024-11-28T02:06:40.487ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
67422024-11-28T02:06:40.487ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
67432024-11-28T02:06:40.487ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
67442024-11-28T02:06:40.488ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 9.010206ms iteration = 28 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
67452024-11-28T02:06:40.490ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
67462024-11-28T02:06:40.490ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
67472024-11-28T02:06:40.490ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
67482024-11-28T02:06:40.494ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
67492024-11-28T02:06:40.495ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-313847b3-d670-4167-9b09-9e7d3dfcb2a4.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-313847b3-d670-4167-9b09-9e7d3dfcb2a4.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1666, tv_nsec: 965179849 } })
67502024-11-28T02:06:40.495ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1103
67512024-11-28T02:06:40.495ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-313847b3-d670-4167-9b09-9e7d3dfcb2a4.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-313847b3-d670-4167-9b09-9e7d3dfcb2a4.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1666, tv_nsec: 965610747 } })
67522024-11-28T02:06:40.496ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1103
67532024-11-28T02:06:40.496ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
67542024-11-28T02:06:40.496ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
67552024-11-28T02:06:40.496ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
67562024-11-28T02:06:40.504ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
67572024-11-28T02:06:40.505ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
67582024-11-28T02:06:40.506ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
67592024-11-28T02:06:40.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
67602024-11-28T02:06:40.507ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
67612024-11-28T02:06:40.507ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
67622024-11-28T02:06:40.507ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
67632024-11-28T02:06:40.507ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
67642024-11-28T02:06:40.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
67652024-11-28T02:06:40.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
67662024-11-28T02:06:40.510ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-313847b3-d670-4167-9b09-9e7d3dfcb2a4.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-313847b3-d670-4167-9b09-9e7d3dfcb2a4.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1666, tv_nsec: 980698029 } })
67672024-11-28T02:06:40.511ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1103 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc
67682024-11-28T02:06:40.522ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
67692024-11-28T02:06:40.522ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
67702024-11-28T02:06:40.524ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
67712024-11-28T02:06:40.524ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
67722024-11-28T02:06:40.526ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 46.494977ms iteration = 28 status = {"errors":[],"garbage_collect_requested":[]}
67732024-11-28T02:06:40.530ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
67742024-11-28T02:06:40.530ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
67752024-11-28T02:06:40.532ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
67762024-11-28T02:06:40.532ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
67772024-11-28T02:06:40.533ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
67782024-11-28T02:06:40.534ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
67792024-11-28T02:06:40.534ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
67802024-11-28T02:06:40.534ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
67812024-11-28T02:06:40.534ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
67822024-11-28T02:06:40.534ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
67832024-11-28T02:06:40.535ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
67842024-11-28T02:06:40.535ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
67852024-11-28T02:06:40.535ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
67862024-11-28T02:06:40.535ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
67872024-11-28T02:06:40.535ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
67882024-11-28T02:06:40.536ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
67892024-11-28T02:06:40.536ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 56.510881ms iteration = 28 status = {"errors":[],"records_set_to_done":[]}
67902024-11-28T02:06:40.546ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
67912024-11-28T02:06:40.546ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 66.168289ms iteration = 28 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
67922024-11-28T02:06:40.554ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
67932024-11-28T02:06:40.557ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 67ac247f-c934-4c78-857a-fba4fbeb9937, lookup_type: ById(67ac247f-c934-4c78-857a-fba4fbeb9937) } result = Ok(())
67942024-11-28T02:06:40.557ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
67952024-11-28T02:06:40.558ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
67962024-11-28T02:06:40.564ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 84.811134ms iteration = 82 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
67972024-11-28T02:06:40.568ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
67982024-11-28T02:06:40.568ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
67992024-11-28T02:06:40.580ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
68002024-11-28T02:06:40.581ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
68012024-11-28T02:06:40.603ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
68022024-11-28T02:06:40.603ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
68032024-11-28T02:06:40.604ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "48c29f7a-ab75-4188-8581-1e23775683d9", "content-length": "26", "date": "Thu, 28 Nov 2024 02:06:40 GMT"} })
68042024-11-28T02:06:40.604ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
68052024-11-28T02:06:40.606ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
68062024-11-28T02:06:40.615ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
68072024-11-28T02:06:40.615ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
68082024-11-28T02:06:40.615ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "804f57db-7406-43ef-bfd7-08947f00502b", "content-length": "26", "date": "Thu, 28 Nov 2024 02:06:40 GMT"} })
68092024-11-28T02:06:40.615ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
68102024-11-28T02:06:40.617ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
68112024-11-28T02:06:40.622ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1111 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc zone_address = fd00:1122:3344:101::2
68122024-11-28T02:06:40.622ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
68132024-11-28T02:06:40.622ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "4a74c2f2-2edd-4b9f-9818-3a7a175ca607", "content-length": "26", "date": "Thu, 28 Nov 2024 02:06:40 GMT"} })
68142024-11-28T02:06:40.622ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1116 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
68152024-11-28T02:06:40.624ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc
68162024-11-28T02:06:40.637ZINFOnexus (BackgroundTasks): VPC route manager sled 313847b3-d670-4167-9b09-9e7d3dfcb2a4 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
68172024-11-28T02:06:40.637ZDEBGnexus (BackgroundTasks): client request SledAgent = 313847b3-d670-4167-9b09-9e7d3dfcb2a4 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
68182024-11-28T02:06:40.637ZDEBGnexus (BackgroundTasks): client response SledAgent = 313847b3-d670-4167-9b09-9e7d3dfcb2a4 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "a5ce1fc7-c2c3-42b6-a0ac-2f8c620e8ff0", "content-length": "372", "date": "Thu, 28 Nov 2024 02:06:40 GMT"} })
68192024-11-28T02:06:40.637ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
68202024-11-28T02:06:40.638ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
68212024-11-28T02:06:40.644ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 164.478317ms iteration = 28 status = {"errors":[],"requests_created_ok":[],"start_invoked_ok":[]}
68222024-11-28T02:06:40.644ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
68232024-11-28T02:06:40.645ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
68242024-11-28T02:06:40.647ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
68252024-11-28T02:06:40.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
68262024-11-28T02:06:40.656ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 67ac247f-c934-4c78-857a-fba4fbeb9937 original_time_created = 2024-11-28 01:53:03.159475 UTC target_id = 67ac247f-c934-4c78-857a-fba4fbeb9937 time_created = 2024-11-28 01:53:03.159475 UTC
68272024-11-28T02:06:40.656ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 176.952314ms iteration = 82 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"67ac247f-c934-4c78-857a-fba4fbeb9937","time_created":"2024-11-28T01:53:03.159475Z"}
68282024-11-28T02:06:40.667ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
68292024-11-28T02:06:40.667ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
68302024-11-28T02:06:40.669ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
68312024-11-28T02:06:40.670ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
68322024-11-28T02:06:40.670ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 190.222994ms iteration = 28 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
68332024-11-28T02:06:40.672ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 192.798689ms iteration = 28 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[]}
68342024-11-28T02:06:40.674ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
68352024-11-28T02:06:40.674ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
68362024-11-28T02:06:40.674ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
68372024-11-28T02:06:40.675ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
68382024-11-28T02:06:40.689ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
68392024-11-28T02:06:40.692ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
68402024-11-28T02:06:40.692ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
68412024-11-28T02:06:40.692ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
68422024-11-28T02:06:40.696ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
68432024-11-28T02:06:40.696ZDEBGnexus (BackgroundTasks): client request SledAgent = 313847b3-d670-4167-9b09-9e7d3dfcb2a4 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
68442024-11-28T02:06:40.696ZDEBGnexus (BackgroundTasks): client response SledAgent = 313847b3-d670-4167-9b09-9e7d3dfcb2a4 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "2cb14446-7fd0-4f11-a203-2c2c29e4a0a1", "date": "Thu, 28 Nov 2024 02:06:40 GMT"} })
68452024-11-28T02:06:40.697ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
68462024-11-28T02:06:40.697ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
68472024-11-28T02:06:40.699ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
68482024-11-28T02:06:40.700ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
68492024-11-28T02:06:40.706ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 226.898601ms iteration = 28 status = {}
68502024-11-28T02:06:40.707ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
68512024-11-28T02:06:40.707ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
68522024-11-28T02:06:40.710ZINFOnexus (BackgroundTasks): VPC route manager sled 313847b3-d670-4167-9b09-9e7d3dfcb2a4 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
68532024-11-28T02:06:40.710ZDEBGnexus (BackgroundTasks): client request SledAgent = 313847b3-d670-4167-9b09-9e7d3dfcb2a4 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
68542024-11-28T02:06:40.711ZDEBGnexus (BackgroundTasks): client response SledAgent = 313847b3-d670-4167-9b09-9e7d3dfcb2a4 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "c12134a8-1d83-4bc8-8649-73d280d13ff0", "date": "Thu, 28 Nov 2024 02:06:40 GMT"} })
68552024-11-28T02:06:40.716ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 237.074618ms iteration = 30 status = {}
68562024-11-28T02:06:40.723ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
68572024-11-28T02:06:40.726ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 313847b3-d670-4167-9b09-9e7d3dfcb2a4, lookup_type: ById(313847b3-d670-4167-9b09-9e7d3dfcb2a4) } result = Ok(())
68582024-11-28T02:06:40.726ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
68592024-11-28T02:06:40.726ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
68602024-11-28T02:06:40.727ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
68612024-11-28T02:06:40.727ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
68622024-11-28T02:06:40.727ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 247.799574ms iteration = 28 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
68632024-11-28T02:06:40.728ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 248.760301ms iteration = 28 status = 0
68642024-11-28T02:06:40.765ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
68652024-11-28T02:06:40.765ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
68662024-11-28T02:06:40.767ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
68672024-11-28T02:06:40.770ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
68682024-11-28T02:06:40.770ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
68692024-11-28T02:06:40.770ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
68702024-11-28T02:06:40.789ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
68712024-11-28T02:06:40.789ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "52ee6b71-80dd-4b33-a9c8-6774e1b76997", "content-length": "2", "date": "Thu, 28 Nov 2024 02:06:40 GMT"} })
68722024-11-28T02:06:40.792ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
68732024-11-28T02:06:40.793ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
68742024-11-28T02:06:40.795ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 315.512961ms iteration = 28 status = null
68752024-11-28T02:06:40.809ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
68762024-11-28T02:06:40.812ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
68772024-11-28T02:06:40.812ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
68782024-11-28T02:06:40.812ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
68792024-11-28T02:06:40.830ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 350.529059ms iteration = 32 status = {"physical_disks_added":0}
68802024-11-28T02:06:40.866ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
68812024-11-28T02:06:40.867ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
68822024-11-28T02:06:40.868ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 4bb2986f-2e92-4ac3-9dc1-591b6bb60b3e, rack_id: 92a8ff6e-21d3-4c28-bcad-e323847985dc, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(ea300318-9d62-42ed-80ea-21c86590a166) } rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc switch_location = Switch0
68832024-11-28T02:06:40.868ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
68842024-11-28T02:06:40.868ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
68852024-11-28T02:06:40.879ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
68862024-11-28T02:06:40.879ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "d04f77e6-1b8b-4306-ad75-d2cb051bad9d", "content-length": "134", "date": "Thu, 28 Nov 2024 02:06:40 GMT"} })
68872024-11-28T02:06:40.879ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1746 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
68882024-11-28T02:06:40.879ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1773 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = ea300318-9d62-42ed-80ea-21c86590a166
68892024-11-28T02:06:40.880ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
68902024-11-28T02:06:40.880ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
68912024-11-28T02:06:40.880ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
68922024-11-28T02:06:40.880ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
68932024-11-28T02:06:40.880ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
68942024-11-28T02:06:40.880ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1920 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
68952024-11-28T02:06:40.880ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
68962024-11-28T02:06:40.880ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
68972024-11-28T02:06:40.880ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc
68982024-11-28T02:06:40.880ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
68992024-11-28T02:06:40.880ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
69002024-11-28T02:06:40.882ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
69012024-11-28T02:06:40.882ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "f78ad2d2-3e90-48ec-ac89-eaef9110041a", "content-length": "2", "date": "Thu, 28 Nov 2024 02:06:40 GMT"} })
69022024-11-28T02:06:40.882ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
69032024-11-28T02:06:40.882ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "9e84df37-34cf-44c1-b072-037a6afae96f", "content-length": "52", "date": "Thu, 28 Nov 2024 02:06:40 GMT"} })
69042024-11-28T02:06:40.882ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc switch_location = Switch0
69052024-11-28T02:06:40.882ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
69062024-11-28T02:06:40.904ZDEBGnexus (BackgroundTasks): client request SledAgent = 313847b3-d670-4167-9b09-9e7d3dfcb2a4 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
69072024-11-28T02:06:40.906ZDEBGnexus (BackgroundTasks): client response SledAgent = 313847b3-d670-4167-9b09-9e7d3dfcb2a4 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "00cd8a05-31c4-4f26-abe0-6a1c23b7ae77", "content-length": "691", "date": "Thu, 28 Nov 2024 02:06:40 GMT"} })
69082024-11-28T02:06:40.906ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
69092024-11-28T02:06:40.906ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
69102024-11-28T02:06:40.906ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
69112024-11-28T02:06:40.907ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 427.902673ms iteration = 29 status = {}
69122024-11-28T02:06:40.984ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "811395e1-d45e-4b81-9d92-2e3f5f0da866", "date": "Thu, 28 Nov 2024 02:06:40 GMT"} })
69132024-11-28T02:06:40.984ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc switch = Switch0
69142024-11-28T02:06:40.984ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
69152024-11-28T02:06:40.984ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "d29150df-9382-4f9a-91a2-ba01bec6c58f", "content-length": "682", "date": "Thu, 28 Nov 2024 02:06:40 GMT"} })
69162024-11-28T02:06:40.984ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
69172024-11-28T02:06:40.984ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
69182024-11-28T02:06:40.990ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
69192024-11-28T02:06:40.991ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
69202024-11-28T02:06:40.994ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
69212024-11-28T02:06:40.994ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
69222024-11-28T02:06:40.996ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
69232024-11-28T02:06:40.997ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
69242024-11-28T02:06:41.004ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 524.45385ms iteration = 28 status = {}
69252024-11-28T02:06:41.212ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 73c7bd4c-878d-45b7-8ff1-361986e76bde roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
69262024-11-28T02:06:41.213ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 73c7bd4c-878d-45b7-8ff1-361986e76bde resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
69272024-11-28T02:06:41.215ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2299 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 73c7bd4c-878d-45b7-8ff1-361986e76bde response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
69282024-11-28T02:06:42.234ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = e4c83ffa-454b-412b-bf49-a8194ec85b3c roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
69292024-11-28T02:06:42.234ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = e4c83ffa-454b-412b-bf49-a8194ec85b3c resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
69302024-11-28T02:06:42.236ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2521 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = e4c83ffa-454b-412b-bf49-a8194ec85b3c response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
69312024-11-28T02:06:43.256ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = f8170bf4-905e-4af8-afc4-adacbd09cde3 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
69322024-11-28T02:06:43.256ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = f8170bf4-905e-4af8-afc4-adacbd09cde3 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
69332024-11-28T02:06:43.258ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2381 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = f8170bf4-905e-4af8-afc4-adacbd09cde3 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
69342024-11-28T02:06:44.195ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1515 local_addr = [fd00:1122:3344:101::b]:55621 method = GET remote_addr = [fd00:1122:3344:101::d]:56417 req_id = 7c0c3ae7-cb60-4fab-b5a3-a126b28e063d response_code = 200 uri = /2469a748-bcd2-4e0d-99cc-f79def0452cc
69352024-11-28T02:06:44.277ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 4ae5afbc-32de-43d9-aaaf-b5f7333b3d6c roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
69362024-11-28T02:06:44.278ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 4ae5afbc-32de-43d9-aaaf-b5f7333b3d6c resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
69372024-11-28T02:06:44.280ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2924 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 4ae5afbc-32de-43d9-aaaf-b5f7333b3d6c response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
69382024-11-28T02:06:45.298ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 1816f362-f7f5-431e-bbb7-720a1ac85a68 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
69392024-11-28T02:06:45.299ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 1816f362-f7f5-431e-bbb7-720a1ac85a68 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
69402024-11-28T02:06:45.301ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2628 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 1816f362-f7f5-431e-bbb7-720a1ac85a68 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
69412024-11-28T02:06:46.319ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 3a86d99d-99be-4a3f-9a8f-4c19c06eed44 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
69422024-11-28T02:06:46.320ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 3a86d99d-99be-4a3f-9a8f-4c19c06eed44 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
69432024-11-28T02:06:46.321ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2336 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 3a86d99d-99be-4a3f-9a8f-4c19c06eed44 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
69442024-11-28T02:06:47.339ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 040400e3-d88c-4a30-a720-b71330217d17 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
69452024-11-28T02:06:47.340ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 040400e3-d88c-4a30-a720-b71330217d17 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
69462024-11-28T02:06:47.342ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2558 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 040400e3-d88c-4a30-a720-b71330217d17 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
69472024-11-28T02:06:48.361ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = cc1e4e84-7cff-43d5-b4f1-6e8f71d19749 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
69482024-11-28T02:06:48.362ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = cc1e4e84-7cff-43d5-b4f1-6e8f71d19749 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
69492024-11-28T02:06:48.363ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2102 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = cc1e4e84-7cff-43d5-b4f1-6e8f71d19749 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
69502024-11-28T02:06:49.381ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 66391c2d-f134-403b-a242-796d41018ba2 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
69512024-11-28T02:06:49.382ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 66391c2d-f134-403b-a242-796d41018ba2 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
69522024-11-28T02:06:49.383ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2332 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 66391c2d-f134-403b-a242-796d41018ba2 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
69532024-11-28T02:06:50.402ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 2fbe4873-4b4e-45f8-80fc-63ca9e09fac2 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
69542024-11-28T02:06:50.402ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 2fbe4873-4b4e-45f8-80fc-63ca9e09fac2 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
69552024-11-28T02:06:50.404ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2019 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 2fbe4873-4b4e-45f8-80fc-63ca9e09fac2 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
69562024-11-28T02:06:50.479ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 83 reason = Timeout
69572024-11-28T02:06:50.479ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 83 reason = Timeout
69582024-11-28T02:06:50.479ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
69592024-11-28T02:06:50.479ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
69602024-11-28T02:06:50.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
69612024-11-28T02:06:50.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
69622024-11-28T02:06:50.481ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
69632024-11-28T02:06:50.482ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
69642024-11-28T02:06:50.482ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
69652024-11-28T02:06:50.483ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
69662024-11-28T02:06:50.483ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
69672024-11-28T02:06:50.483ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
69682024-11-28T02:06:50.485ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
69692024-11-28T02:06:50.485ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
69702024-11-28T02:06:50.517ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 38.070291ms iteration = 83 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
69712024-11-28T02:06:50.521ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
69722024-11-28T02:06:50.524ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 67ac247f-c934-4c78-857a-fba4fbeb9937, lookup_type: ById(67ac247f-c934-4c78-857a-fba4fbeb9937) } result = Ok(())
69732024-11-28T02:06:50.524ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
69742024-11-28T02:06:50.524ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
69752024-11-28T02:06:50.536ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 67ac247f-c934-4c78-857a-fba4fbeb9937 original_time_created = 2024-11-28 01:53:03.159475 UTC target_id = 67ac247f-c934-4c78-857a-fba4fbeb9937 time_created = 2024-11-28 01:53:03.159475 UTC
69762024-11-28T02:06:50.536ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 57.279865ms iteration = 83 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"67ac247f-c934-4c78-857a-fba4fbeb9937","time_created":"2024-11-28T01:53:03.159475Z"}
69772024-11-28T02:06:51.423ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = ec0e13bf-9c5b-4d63-b33c-73406423029a roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
69782024-11-28T02:06:51.424ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = ec0e13bf-9c5b-4d63-b33c-73406423029a resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
69792024-11-28T02:06:51.426ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2234 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = ec0e13bf-9c5b-4d63-b33c-73406423029a response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
69802024-11-28T02:06:52.445ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 8ee04216-d940-4db2-8ece-488d8cfd5996 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
69812024-11-28T02:06:52.445ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 8ee04216-d940-4db2-8ece-488d8cfd5996 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
69822024-11-28T02:06:52.447ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2303 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 8ee04216-d940-4db2-8ece-488d8cfd5996 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
69832024-11-28T02:06:53.465ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 7036694f-164f-49e6-9478-814675c7b1b3 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
69842024-11-28T02:06:53.466ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 7036694f-164f-49e6-9478-814675c7b1b3 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
69852024-11-28T02:06:53.468ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2549 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 7036694f-164f-49e6-9478-814675c7b1b3 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
69862024-11-28T02:06:54.186ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:55495 req_id = 81eb4ba6-b040-43e8-bcb4-881560baf2b3 roles = RoleSet { roles: {} } uri = /metrics/collectors/119eed64-2e24-4d5f-8c42-be6cd8bec8b5/producers?sort_by=id_ascending
69872024-11-28T02:06:54.186ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:55495 req_id = 81eb4ba6-b040-43e8-bcb4-881560baf2b3 resource = Database result = Ok(()) uri = /metrics/collectors/119eed64-2e24-4d5f-8c42-be6cd8bec8b5/producers?sort_by=id_ascending
69882024-11-28T02:06:54.189ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2842 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:55495 req_id = 81eb4ba6-b040-43e8-bcb4-881560baf2b3 response_code = 200 uri = /metrics/collectors/119eed64-2e24-4d5f-8c42-be6cd8bec8b5/producers?sort_by=id_ascending
69892024-11-28T02:06:54.189ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:55495 req_id = e606375e-5d92-443a-b626-f618ba1a367d roles = RoleSet { roles: {} } uri = /metrics/collectors/119eed64-2e24-4d5f-8c42-be6cd8bec8b5/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJjNDM0YjQ0OC01MTk5LTQ2NmUtYWYzMi00ZjYyOWUzMWM3YjEifX0%3D
69902024-11-28T02:06:54.189ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:55495 req_id = e606375e-5d92-443a-b626-f618ba1a367d resource = Database result = Ok(()) uri = /metrics/collectors/119eed64-2e24-4d5f-8c42-be6cd8bec8b5/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJjNDM0YjQ0OC01MTk5LTQ2NmUtYWYzMi00ZjYyOWUzMWM3YjEifX0%3D
69912024-11-28T02:06:54.191ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2440 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:55495 req_id = e606375e-5d92-443a-b626-f618ba1a367d response_code = 200 uri = /metrics/collectors/119eed64-2e24-4d5f-8c42-be6cd8bec8b5/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJjNDM0YjQ0OC01MTk5LTQ2NmUtYWYzMi00ZjYyOWUzMWM3YjEifX0%3D
69922024-11-28T02:06:54.195ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1282 local_addr = [fd00:1122:3344:101::b]:55621 method = GET remote_addr = [fd00:1122:3344:101::d]:56417 req_id = 12a87011-4053-4aed-b90e-3867986986ff response_code = 200 uri = /2469a748-bcd2-4e0d-99cc-f79def0452cc
69932024-11-28T02:06:54.486ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 3d64480e-78cf-48bc-b3c5-8e7b54b2cfb1 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
69942024-11-28T02:06:54.486ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 3d64480e-78cf-48bc-b3c5-8e7b54b2cfb1 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
69952024-11-28T02:06:54.488ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2304 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 3d64480e-78cf-48bc-b3c5-8e7b54b2cfb1 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
69962024-11-28T02:06:55.508ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 6f8c67ee-67bf-4688-8b0f-6f35b42e88d8 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
69972024-11-28T02:06:55.509ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 6f8c67ee-67bf-4688-8b0f-6f35b42e88d8 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
69982024-11-28T02:06:55.511ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2514 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 6f8c67ee-67bf-4688-8b0f-6f35b42e88d8 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
69992024-11-28T02:06:56.531ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 27f6c6d0-8af3-44ad-ab59-dec4688a595d roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
70002024-11-28T02:06:56.531ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 27f6c6d0-8af3-44ad-ab59-dec4688a595d resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
70012024-11-28T02:06:56.533ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2322 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 27f6c6d0-8af3-44ad-ab59-dec4688a595d response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
70022024-11-28T02:06:57.552ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = e09799a6-79b9-4010-929b-8b511ce24431 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
70032024-11-28T02:06:57.553ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = e09799a6-79b9-4010-929b-8b511ce24431 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
70042024-11-28T02:06:57.555ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2563 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = e09799a6-79b9-4010-929b-8b511ce24431 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
70052024-11-28T02:06:58.574ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 71cfd34a-31a3-4a4f-900c-2db3f282087b roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
70062024-11-28T02:06:58.574ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 71cfd34a-31a3-4a4f-900c-2db3f282087b resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
70072024-11-28T02:06:58.576ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1956 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 71cfd34a-31a3-4a4f-900c-2db3f282087b response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
70082024-11-28T02:06:59.594ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 81f6a584-a306-43ce-a2ed-f1fde3954ad8 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
70092024-11-28T02:06:59.595ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 81f6a584-a306-43ce-a2ed-f1fde3954ad8 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
70102024-11-28T02:06:59.596ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1905 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 81f6a584-a306-43ce-a2ed-f1fde3954ad8 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
70112024-11-28T02:07:00.479ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 84 reason = Timeout
70122024-11-28T02:07:00.479ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 84 reason = Timeout
70132024-11-28T02:07:00.479ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
70142024-11-28T02:07:00.479ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
70152024-11-28T02:07:00.480ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
70162024-11-28T02:07:00.480ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
70172024-11-28T02:07:00.506ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
70182024-11-28T02:07:00.507ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
70192024-11-28T02:07:00.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
70202024-11-28T02:07:00.509ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
70212024-11-28T02:07:00.509ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
70222024-11-28T02:07:00.509ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
70232024-11-28T02:07:00.511ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
70242024-11-28T02:07:00.511ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
70252024-11-28T02:07:00.513ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
70262024-11-28T02:07:00.517ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 67ac247f-c934-4c78-857a-fba4fbeb9937, lookup_type: ById(67ac247f-c934-4c78-857a-fba4fbeb9937) } result = Ok(())
70272024-11-28T02:07:00.517ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
70282024-11-28T02:07:00.517ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
70292024-11-28T02:07:00.529ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 67ac247f-c934-4c78-857a-fba4fbeb9937 original_time_created = 2024-11-28 01:53:03.159475 UTC target_id = 67ac247f-c934-4c78-857a-fba4fbeb9937 time_created = 2024-11-28 01:53:03.159475 UTC
70302024-11-28T02:07:00.529ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 49.549061ms iteration = 84 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"67ac247f-c934-4c78-857a-fba4fbeb9937","time_created":"2024-11-28T01:53:03.159475Z"}
70312024-11-28T02:07:00.542ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 63.132006ms iteration = 84 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
70322024-11-28T02:07:00.614ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = cd7fcfdc-7d39-4f3a-ba11-915694f75cf9 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
70332024-11-28T02:07:00.615ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = cd7fcfdc-7d39-4f3a-ba11-915694f75cf9 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
70342024-11-28T02:07:00.616ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1917 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = cd7fcfdc-7d39-4f3a-ba11-915694f75cf9 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
70352024-11-28T02:07:01.634ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 96652a76-029c-47ac-8f07-f562f25bc30b roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
70362024-11-28T02:07:01.635ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 96652a76-029c-47ac-8f07-f562f25bc30b resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
70372024-11-28T02:07:01.637ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2687 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 96652a76-029c-47ac-8f07-f562f25bc30b response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
70382024-11-28T02:07:02.656ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = f2cfa153-80fa-451c-8380-2c41de34e9cd roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
70392024-11-28T02:07:02.657ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = f2cfa153-80fa-451c-8380-2c41de34e9cd resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
70402024-11-28T02:07:02.659ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2355 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = f2cfa153-80fa-451c-8380-2c41de34e9cd response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
70412024-11-28T02:07:03.678ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 5ed90c08-bd2a-45c6-bfcf-e93a3fa36032 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
70422024-11-28T02:07:03.678ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 5ed90c08-bd2a-45c6-bfcf-e93a3fa36032 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
70432024-11-28T02:07:03.680ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2295 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 5ed90c08-bd2a-45c6-bfcf-e93a3fa36032 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
70442024-11-28T02:07:04.195ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1653 local_addr = [fd00:1122:3344:101::b]:55621 method = GET remote_addr = [fd00:1122:3344:101::d]:56417 req_id = b7e68a86-1f8f-41c2-ba8e-7f4c87047cb5 response_code = 200 uri = /2469a748-bcd2-4e0d-99cc-f79def0452cc
70452024-11-28T02:07:04.699ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = aa5dd97e-fc4f-419f-bc8a-3671d55632cb roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
70462024-11-28T02:07:04.700ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = aa5dd97e-fc4f-419f-bc8a-3671d55632cb resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
70472024-11-28T02:07:04.702ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2332 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = aa5dd97e-fc4f-419f-bc8a-3671d55632cb response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
70482024-11-28T02:07:05.720ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 5ebe859c-eaed-4808-a718-ff919805389a roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
70492024-11-28T02:07:05.721ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 5ebe859c-eaed-4808-a718-ff919805389a resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
70502024-11-28T02:07:05.722ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2115 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 5ebe859c-eaed-4808-a718-ff919805389a response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
70512024-11-28T02:07:06.742ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = ff843e38-c6d8-423a-bbdd-760dd78c677f roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
70522024-11-28T02:07:06.742ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = ff843e38-c6d8-423a-bbdd-760dd78c677f resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
70532024-11-28T02:07:06.744ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2575 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = ff843e38-c6d8-423a-bbdd-760dd78c677f response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
70542024-11-28T02:07:07.765ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = a9a432f6-bd3a-4055-a112-4d071abfe552 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
70552024-11-28T02:07:07.765ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = a9a432f6-bd3a-4055-a112-4d071abfe552 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
70562024-11-28T02:07:07.767ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1942 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = a9a432f6-bd3a-4055-a112-4d071abfe552 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
70572024-11-28T02:07:08.644ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:40402 req_id = ffb3f640-e2f2-4e7a-9254-81b08b2f11f3 roles = RoleSet { roles: {} } uri = /nat/ipv4/changeset/10?limit=100
70582024-11-28T02:07:08.644ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:40402 req_id = ffb3f640-e2f2-4e7a-9254-81b08b2f11f3 resource = Database result = Ok(()) uri = /nat/ipv4/changeset/10?limit=100
70592024-11-28T02:07:08.647ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 3532 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:40402 req_id = ffb3f640-e2f2-4e7a-9254-81b08b2f11f3 response_code = 200 uri = /nat/ipv4/changeset/10?limit=100
70602024-11-28T02:07:08.785ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 063f19e6-9e6c-470c-9f26-13d98e6f802a roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
70612024-11-28T02:07:08.786ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 063f19e6-9e6c-470c-9f26-13d98e6f802a resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
70622024-11-28T02:07:08.787ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2393 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 063f19e6-9e6c-470c-9f26-13d98e6f802a response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
70632024-11-28T02:07:09.806ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 642a3d3c-9cef-4400-ae0f-7b6f5bcdfa21 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
70642024-11-28T02:07:09.806ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 642a3d3c-9cef-4400-ae0f-7b6f5bcdfa21 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
70652024-11-28T02:07:09.808ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2176 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 642a3d3c-9cef-4400-ae0f-7b6f5bcdfa21 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
70662024-11-28T02:07:10.479ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 17 reason = Timeout
70672024-11-28T02:07:10.479ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 29 reason = Timeout
70682024-11-28T02:07:10.479ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
70692024-11-28T02:07:10.479ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 16 reason = Timeout
70702024-11-28T02:07:10.479ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
70712024-11-28T02:07:10.479ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
70722024-11-28T02:07:10.479ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 85 reason = Timeout
70732024-11-28T02:07:10.479ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 29 reason = Timeout
70742024-11-28T02:07:10.479ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
70752024-11-28T02:07:10.479ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
70762024-11-28T02:07:10.479ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 33 reason = Timeout
70772024-11-28T02:07:10.479ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 16 reason = Timeout
70782024-11-28T02:07:10.479ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 15 reason = Timeout
70792024-11-28T02:07:10.479ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
70802024-11-28T02:07:10.479ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:56
70812024-11-28T02:07:10.479ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
70822024-11-28T02:07:10.479ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
70832024-11-28T02:07:10.479ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 15 reason = Timeout
70842024-11-28T02:07:10.479ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
70852024-11-28T02:07:10.479ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 29 reason = Timeout
70862024-11-28T02:07:10.479ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
70872024-11-28T02:07:10.480ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 16 reason = Timeout
70882024-11-28T02:07:10.480ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
70892024-11-28T02:07:10.480ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
70902024-11-28T02:07:10.480ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 85 reason = Timeout
70912024-11-28T02:07:10.480ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
70922024-11-28T02:07:10.480ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 15 reason = Timeout
70932024-11-28T02:07:10.480ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2024-11-28 01:57:10.479932651 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
70942024-11-28T02:07:10.480ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
70952024-11-28T02:07:10.480ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 29 reason = Timeout
70962024-11-28T02:07:10.480ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
70972024-11-28T02:07:10.480ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 17 reason = Timeout
70982024-11-28T02:07:10.480ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 15 reason = Timeout
70992024-11-28T02:07:10.480ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
71002024-11-28T02:07:10.480ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 15 reason = Timeout
71012024-11-28T02:07:10.480ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
71022024-11-28T02:07:10.480ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 29 reason = Timeout
71032024-11-28T02:07:10.480ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
71042024-11-28T02:07:10.480ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 16 reason = Timeout
71052024-11-28T02:07:10.480ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:83 reason = blueprint disabled target_id = 67ac247f-c934-4c78-857a-fba4fbeb9937
71062024-11-28T02:07:10.480ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 29 reason = Timeout
71072024-11-28T02:07:10.480ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 15.49\u{b5}s iteration = 16 status = {"enabled":false,"target_id":"67ac247f-c934-4c78-857a-fba4fbeb9937"}
71082024-11-28T02:07:10.480ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 29 reason = Timeout
71092024-11-28T02:07:10.480ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 31 reason = Timeout
71102024-11-28T02:07:10.480ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
71112024-11-28T02:07:10.480ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
71122024-11-28T02:07:10.480ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 29 reason = Timeout
71132024-11-28T02:07:10.480ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
71142024-11-28T02:07:10.480ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
71152024-11-28T02:07:10.480ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 29 reason = Timeout
71162024-11-28T02:07:10.480ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 29 reason = Timeout
71172024-11-28T02:07:10.480ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
71182024-11-28T02:07:10.480ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
71192024-11-28T02:07:10.480ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
71202024-11-28T02:07:10.480ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 29 reason = Timeout
71212024-11-28T02:07:10.480ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
71222024-11-28T02:07:10.480ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 30 reason = Timeout
71232024-11-28T02:07:10.480ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
71242024-11-28T02:07:10.480ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 16 reason = Timeout
71252024-11-28T02:07:10.480ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 29 reason = Timeout
71262024-11-28T02:07:10.480ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
71272024-11-28T02:07:10.480ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 16 reason = Timeout
71282024-11-28T02:07:10.480ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
71292024-11-28T02:07:10.480ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
71302024-11-28T02:07:10.480ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
71312024-11-28T02:07:10.480ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
71322024-11-28T02:07:10.480ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
71332024-11-28T02:07:10.480ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
71342024-11-28T02:07:10.480ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
71352024-11-28T02:07:10.480ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
71362024-11-28T02:07:10.480ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
71372024-11-28T02:07:10.480ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
71382024-11-28T02:07:10.480ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
71392024-11-28T02:07:10.480ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
71402024-11-28T02:07:10.480ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
71412024-11-28T02:07:10.480ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
71422024-11-28T02:07:10.480ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
71432024-11-28T02:07:10.480ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
71442024-11-28T02:07:10.480ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
71452024-11-28T02:07:10.480ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
71462024-11-28T02:07:10.480ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
71472024-11-28T02:07:10.480ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
71482024-11-28T02:07:10.480ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
71492024-11-28T02:07:10.480ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
71502024-11-28T02:07:10.480ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
71512024-11-28T02:07:10.480ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
71522024-11-28T02:07:10.480ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("41ea4a99-3038-474e-bf28-2ad338f3243f.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("9402a51b-d30f-4544-93ec-0f5dc7a485e4.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("d23bc5f2-6b92-4992-bdd2-8695e5bf9c0f.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("d23bc5f2-6b92-4992-bdd2-8695e5bf9c0f.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:1::1))) }], valid_until: Instant { tv_sec: 1696, tv_nsec: 950579986 } })
71532024-11-28T02:07:10.480ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("36d4f615-7461-4364-960c-89ece3aaabaf.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("eb274350-c0b0-4bce-9c8a-502880f2c905.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("eb274350-c0b0-4bce-9c8a-502880f2c905.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::8))) }], valid_until: Instant { tv_sec: 1696, tv_nsec: 950641597 } })
71542024-11-28T02:07:10.481ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } dns_group = external
71552024-11-28T02:07:10.481ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.1006ms iteration = 16 status = {"addresses":["[fd00:1122:3344:101::9]:5353","[fd00:1122:3344:101::8]:5353"]}
71562024-11-28T02:07:10.481ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } dns_group = internal
71572024-11-28T02:07:10.481ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.383034ms iteration = 16 status = {"addresses":["[fd00:1122:3344:3::1]:5353","[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:1::1]:5353"]}
71582024-11-28T02:07:10.481ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
71592024-11-28T02:07:10.481ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
71602024-11-28T02:07:10.483ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
71612024-11-28T02:07:10.483ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.802766ms iteration = 29 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
71622024-11-28T02:07:10.484ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
71632024-11-28T02:07:10.485ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
71642024-11-28T02:07:10.486ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2024-11-28 01:57:10.479932651 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
71652024-11-28T02:07:10.486ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 6.95894ms iteration = 15 status = {"expiration":"2024-11-28T01:57:10.479932651Z","pruned":[]}
71662024-11-28T02:07:10.487ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 7.724664ms iteration = 15 status = {"errors":[],"found_port_ok":[]}
71672024-11-28T02:07:10.487ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
71682024-11-28T02:07:10.487ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
71692024-11-28T02:07:10.487ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
71702024-11-28T02:07:10.492ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
71712024-11-28T02:07:10.494ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-313847b3-d670-4167-9b09-9e7d3dfcb2a4.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-313847b3-d670-4167-9b09-9e7d3dfcb2a4.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1696, tv_nsec: 963703074 } })
71722024-11-28T02:07:10.495ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1103
71732024-11-28T02:07:10.496ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
71742024-11-28T02:07:10.496ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
71752024-11-28T02:07:10.497ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
71762024-11-28T02:07:10.498ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
71772024-11-28T02:07:10.501ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-313847b3-d670-4167-9b09-9e7d3dfcb2a4.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-313847b3-d670-4167-9b09-9e7d3dfcb2a4.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1696, tv_nsec: 971183684 } })
71782024-11-28T02:07:10.501ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1103
71792024-11-28T02:07:10.502ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
71802024-11-28T02:07:10.502ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
71812024-11-28T02:07:10.502ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
71822024-11-28T02:07:10.505ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
71832024-11-28T02:07:10.510ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
71842024-11-28T02:07:10.512ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
71852024-11-28T02:07:10.512ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
71862024-11-28T02:07:10.524ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
71872024-11-28T02:07:10.524ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
71882024-11-28T02:07:10.536ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
71892024-11-28T02:07:10.539ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
71902024-11-28T02:07:10.539ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
71912024-11-28T02:07:10.539ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
71922024-11-28T02:07:10.540ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
71932024-11-28T02:07:10.540ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
71942024-11-28T02:07:10.541ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
71952024-11-28T02:07:10.542ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
71962024-11-28T02:07:10.542ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
71972024-11-28T02:07:10.543ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
71982024-11-28T02:07:10.543ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
71992024-11-28T02:07:10.545ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
72002024-11-28T02:07:10.545ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
72012024-11-28T02:07:10.545ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
72022024-11-28T02:07:10.545ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
72032024-11-28T02:07:10.546ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
72042024-11-28T02:07:10.546ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
72052024-11-28T02:07:10.547ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
72062024-11-28T02:07:10.548ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
72072024-11-28T02:07:10.548ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:171 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
72082024-11-28T02:07:10.548ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 68.090603ms iteration = 15 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
72092024-11-28T02:07:10.548ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
72102024-11-28T02:07:10.548ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
72112024-11-28T02:07:10.549ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
72122024-11-28T02:07:10.549ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
72132024-11-28T02:07:10.549ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
72142024-11-28T02:07:10.549ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
72152024-11-28T02:07:10.549ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
72162024-11-28T02:07:10.551ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
72172024-11-28T02:07:10.551ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
72182024-11-28T02:07:10.551ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
72192024-11-28T02:07:10.554ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
72202024-11-28T02:07:10.555ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-313847b3-d670-4167-9b09-9e7d3dfcb2a4.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-313847b3-d670-4167-9b09-9e7d3dfcb2a4.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1697, tv_nsec: 24806605 } })
72212024-11-28T02:07:10.555ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 75.470121ms iteration = 29 status = {"errors":[],"requests_created_ok":[],"start_invoked_ok":[]}
72222024-11-28T02:07:10.555ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1103 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc
72232024-11-28T02:07:10.555ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
72242024-11-28T02:07:10.556ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
72252024-11-28T02:07:10.556ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
72262024-11-28T02:07:10.556ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
72272024-11-28T02:07:10.556ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
72282024-11-28T02:07:10.557ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 76.971347ms iteration = 29 status = {"errors":[],"garbage_collect_requested":[]}
72292024-11-28T02:07:10.557ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
72302024-11-28T02:07:10.557ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
72312024-11-28T02:07:10.558ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
72322024-11-28T02:07:10.558ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
72332024-11-28T02:07:10.558ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
72342024-11-28T02:07:10.558ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 78.407472ms iteration = 29 status = {"errors":[],"records_set_to_done":[]}
72352024-11-28T02:07:10.560ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
72362024-11-28T02:07:10.560ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
72372024-11-28T02:07:10.560ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
72382024-11-28T02:07:10.560ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
72392024-11-28T02:07:10.560ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
72402024-11-28T02:07:10.560ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
72412024-11-28T02:07:10.560ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
72422024-11-28T02:07:10.560ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
72432024-11-28T02:07:10.560ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
72442024-11-28T02:07:10.561ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
72452024-11-28T02:07:10.563ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
72462024-11-28T02:07:10.563ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
72472024-11-28T02:07:10.563ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
72482024-11-28T02:07:10.563ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
72492024-11-28T02:07:10.563ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
72502024-11-28T02:07:10.564ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
72512024-11-28T02:07:10.564ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
72522024-11-28T02:07:10.568ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
72532024-11-28T02:07:10.568ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 67ac247f-c934-4c78-857a-fba4fbeb9937, lookup_type: ById(67ac247f-c934-4c78-857a-fba4fbeb9937) } result = Ok(())
72542024-11-28T02:07:10.568ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
72552024-11-28T02:07:10.569ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
72562024-11-28T02:07:10.572ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
72572024-11-28T02:07:10.572ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
72582024-11-28T02:07:10.572ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
72592024-11-28T02:07:10.574ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
72602024-11-28T02:07:10.574ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 94.305489ms iteration = 29 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
72612024-11-28T02:07:10.582ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
72622024-11-28T02:07:10.582ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
72632024-11-28T02:07:10.583ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 67ac247f-c934-4c78-857a-fba4fbeb9937 original_time_created = 2024-11-28 01:53:03.159475 UTC target_id = 67ac247f-c934-4c78-857a-fba4fbeb9937 time_created = 2024-11-28 01:53:03.159475 UTC
72642024-11-28T02:07:10.583ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 103.295304ms iteration = 85 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"67ac247f-c934-4c78-857a-fba4fbeb9937","time_created":"2024-11-28T01:53:03.159475Z"}
72652024-11-28T02:07:10.585ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
72662024-11-28T02:07:10.585ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
72672024-11-28T02:07:10.585ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
72682024-11-28T02:07:10.586ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
72692024-11-28T02:07:10.586ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
72702024-11-28T02:07:10.591ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 313847b3-d670-4167-9b09-9e7d3dfcb2a4, lookup_type: ById(313847b3-d670-4167-9b09-9e7d3dfcb2a4) } result = Ok(())
72712024-11-28T02:07:10.591ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
72722024-11-28T02:07:10.591ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
72732024-11-28T02:07:10.593ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 113.261827ms iteration = 29 status = 0
72742024-11-28T02:07:10.598ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 118.397427ms iteration = 85 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
72752024-11-28T02:07:10.604ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
72762024-11-28T02:07:10.606ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
72772024-11-28T02:07:10.606ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
72782024-11-28T02:07:10.607ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
72792024-11-28T02:07:10.607ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
72802024-11-28T02:07:10.607ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
72812024-11-28T02:07:10.612ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
72822024-11-28T02:07:10.612ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:140 found = 0 not_ready_to_be_deleted = 0
72832024-11-28T02:07:10.612ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 132.531252ms iteration = 15 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
72842024-11-28T02:07:10.615ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
72852024-11-28T02:07:10.615ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
72862024-11-28T02:07:10.615ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 135.534994ms iteration = 29 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
72872024-11-28T02:07:10.615ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 135.548575ms iteration = 29 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
72882024-11-28T02:07:10.619ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
72892024-11-28T02:07:10.619ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
72902024-11-28T02:07:10.621ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
72912024-11-28T02:07:10.621ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
72922024-11-28T02:07:10.621ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
72932024-11-28T02:07:10.621ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
72942024-11-28T02:07:10.621ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
72952024-11-28T02:07:10.621ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
72962024-11-28T02:07:10.623ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
72972024-11-28T02:07:10.623ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
72982024-11-28T02:07:10.624ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
72992024-11-28T02:07:10.627ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
73002024-11-28T02:07:10.627ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 808a324e-c913-4bfc-91a5-d43c961cd511 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
73012024-11-28T02:07:10.627ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
73022024-11-28T02:07:10.627ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 147.512153ms iteration = 29 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[]}
73032024-11-28T02:07:10.627ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
73042024-11-28T02:07:10.648ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
73052024-11-28T02:07:10.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
73062024-11-28T02:07:10.675ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
73072024-11-28T02:07:10.676ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 196.170687ms iteration = 33 status = {"physical_disks_added":0}
73082024-11-28T02:07:10.697ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
73092024-11-28T02:07:10.697ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
73102024-11-28T02:07:10.698ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "ce286b97-aa33-4b75-a1a2-79371e0c451f", "content-length": "26", "date": "Thu, 28 Nov 2024 02:07:10 GMT"} })
73112024-11-28T02:07:10.698ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
73122024-11-28T02:07:10.700ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
73132024-11-28T02:07:10.704ZWARNnexus (BackgroundTasks): instance reincarnation completed with errors background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:103 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0 query_errors = 0 restart_errors = 0
73142024-11-28T02:07:10.704ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 224.574861ms iteration = 15 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
73152024-11-28T02:07:10.746ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
73162024-11-28T02:07:10.821ZINFOnexus (BackgroundTasks): VPC route manager sled 313847b3-d670-4167-9b09-9e7d3dfcb2a4 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
73172024-11-28T02:07:10.821ZDEBGnexus (BackgroundTasks): client request SledAgent = 313847b3-d670-4167-9b09-9e7d3dfcb2a4 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
73182024-11-28T02:07:10.822ZDEBGnexus (BackgroundTasks): client response SledAgent = 313847b3-d670-4167-9b09-9e7d3dfcb2a4 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "c3cf0112-a614-4d3c-8959-bbea53a19ea2", "content-length": "372", "date": "Thu, 28 Nov 2024 02:07:10 GMT"} })
73192024-11-28T02:07:10.822ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
73202024-11-28T02:07:10.822ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
73212024-11-28T02:07:10.823ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
73222024-11-28T02:07:10.824ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
73232024-11-28T02:07:10.827ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
73242024-11-28T02:07:10.827ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
73252024-11-28T02:07:10.827ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 568fa03d-accd-4fb3-8338-0fee0b11712d roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
73262024-11-28T02:07:10.827ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
73272024-11-28T02:07:10.827ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 568fa03d-accd-4fb3-8338-0fee0b11712d resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
73282024-11-28T02:07:10.828ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
73292024-11-28T02:07:10.828ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
73302024-11-28T02:07:10.828ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
73312024-11-28T02:07:10.829ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2516 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 568fa03d-accd-4fb3-8338-0fee0b11712d response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
73322024-11-28T02:07:10.829ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
73332024-11-28T02:07:10.829ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = a7e6af61-408c-483d-b043-c27ccf55708e dns_zone_name = oxide.test found_so_far = 0 version = 2
73342024-11-28T02:07:10.829ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
73352024-11-28T02:07:10.830ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
73362024-11-28T02:07:10.831ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 808a324e-c913-4bfc-91a5-d43c961cd511 dns_zone_name = control-plane.oxide.internal found_so_far = 51 version = 1
73372024-11-28T02:07:10.831ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
73382024-11-28T02:07:10.831ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2024-11-28 01:53:42.502562 UTC dns_group = internal generation = 1
73392024-11-28T02:07:10.831ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 351.943234ms iteration = 16 status = {"generation":1}
73402024-11-28T02:07:10.831ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
73412024-11-28T02:07:10.831ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
73422024-11-28T02:07:10.832ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
73432024-11-28T02:07:10.832ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "344a54c2-d528-470c-9f01-b9ce029948d4", "content-length": "26", "date": "Thu, 28 Nov 2024 02:07:10 GMT"} })
73442024-11-28T02:07:10.832ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
73452024-11-28T02:07:10.834ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
73462024-11-28T02:07:10.834ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
73472024-11-28T02:07:10.834ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
73482024-11-28T02:07:10.835ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
73492024-11-28T02:07:10.837ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = a7e6af61-408c-483d-b043-c27ccf55708e dns_zone_name = oxide.test found_so_far = 1 version = 2
73502024-11-28T02:07:10.837ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
73512024-11-28T02:07:10.837ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2024-11-28 01:53:43.351579 UTC dns_group = external generation = 2
73522024-11-28T02:07:10.837ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 357.580141ms iteration = 16 status = {"generation":2}
73532024-11-28T02:07:10.839ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1111 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc zone_address = fd00:1122:3344:101::2
73542024-11-28T02:07:10.840ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
73552024-11-28T02:07:10.840ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "4e035fbe-c64e-4adc-8155-faca121d982b", "content-length": "26", "date": "Thu, 28 Nov 2024 02:07:10 GMT"} })
73562024-11-28T02:07:10.840ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1116 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
73572024-11-28T02:07:10.841ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
73582024-11-28T02:07:10.841ZDEBGnexus (BackgroundTasks): client request SledAgent = 313847b3-d670-4167-9b09-9e7d3dfcb2a4 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
73592024-11-28T02:07:10.841ZDEBGnexus (BackgroundTasks): client response SledAgent = 313847b3-d670-4167-9b09-9e7d3dfcb2a4 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "1bbb5b10-9f63-4668-aa5a-c2d3b0a90789", "date": "Thu, 28 Nov 2024 02:07:10 GMT"} })
73602024-11-28T02:07:10.841ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
73612024-11-28T02:07:10.842ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
73622024-11-28T02:07:10.842ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc
73632024-11-28T02:07:10.844ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
73642024-11-28T02:07:10.844ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
73652024-11-28T02:07:10.847ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
73662024-11-28T02:07:10.848ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
73672024-11-28T02:07:10.852ZINFOnexus (BackgroundTasks): VPC route manager sled 313847b3-d670-4167-9b09-9e7d3dfcb2a4 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
73682024-11-28T02:07:10.852ZDEBGnexus (BackgroundTasks): client request SledAgent = 313847b3-d670-4167-9b09-9e7d3dfcb2a4 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
73692024-11-28T02:07:10.852ZDEBGnexus (BackgroundTasks): client response SledAgent = 313847b3-d670-4167-9b09-9e7d3dfcb2a4 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "48ff4606-cf35-43d0-92cb-7f3c9a12e38b", "date": "Thu, 28 Nov 2024 02:07:10 GMT"} })
73702024-11-28T02:07:10.854ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 374.247221ms iteration = 31 status = {}
73712024-11-28T02:07:10.858ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2024-11-28T01:53:08.890632Z, time_modified: 2024-11-28T01:53:08.890632Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 7e159834-997a-4c03-b9db-1e91d783221c, db_silo: Silo { identity: SiloIdentity { id: 7e159834-997a-4c03-b9db-1e91d783221c, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-11-28T01:53:43.308696Z, time_modified: 2024-11-28T01:53:43.308696Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "bc39d1b3b4bb84a49d6458ed7aedc5bca9e77742c7b35882b64d188c0602d306" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 7e159834-997a-4c03-b9db-1e91d783221c, db_silo: Silo { identity: SiloIdentity { id: 7e159834-997a-4c03-b9db-1e91d783221c, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-11-28T01:53:43.308696Z, time_modified: 2024-11-28T01:53:43.308696Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "bc39d1b3b4bb84a49d6458ed7aedc5bca9e77742c7b35882b64d188c0602d306" }] }) }
73722024-11-28T02:07:10.858ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 378.763079ms iteration = 16 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"7e159834-997a-4c03-b9db-1e91d783221c","tls_certs":["bc39d1b3b4bb84a49d6458ed7aedc5bca9e77742c7b35882b64d188c0602d306"]}},"default_endpoint":{"silo_id":"7e159834-997a-4c03-b9db-1e91d783221c","tls_certs":["bc39d1b3b4bb84a49d6458ed7aedc5bca9e77742c7b35882b64d188c0602d306"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
73732024-11-28T02:07:10.889ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 408.974594ms iteration = 29 status = {}
73742024-11-28T02:07:10.907ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
73752024-11-28T02:07:10.921ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
73762024-11-28T02:07:10.921ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
73772024-11-28T02:07:10.921ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
73782024-11-28T02:07:10.922ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "95abe8fa-8af3-43d8-8a46-6dadad124d99", "date": "Thu, 28 Nov 2024 02:07:10 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
73792024-11-28T02:07:10.922ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
73802024-11-28T02:07:10.923ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
73812024-11-28T02:07:10.924ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "0a46acb8-1a4f-4304-bf66-9624b4621370", "date": "Thu, 28 Nov 2024 02:07:10 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
73822024-11-28T02:07:10.924ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
73832024-11-28T02:07:10.925ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
73842024-11-28T02:07:10.925ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
73852024-11-28T02:07:10.925ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
73862024-11-28T02:07:10.926ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
73872024-11-28T02:07:10.926ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 447.123826ms iteration = 17 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
73882024-11-28T02:07:10.927ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
73892024-11-28T02:07:10.929ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
73902024-11-28T02:07:10.929ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
73912024-11-28T02:07:10.929ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
73922024-11-28T02:07:10.955ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
73932024-11-28T02:07:10.956ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "bad41db6-9a82-4a10-9785-262732c74931", "content-length": "2", "date": "Thu, 28 Nov 2024 02:07:10 GMT"} })
73942024-11-28T02:07:10.958ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
73952024-11-28T02:07:10.959ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
73962024-11-28T02:07:10.961ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 481.122967ms iteration = 29 status = null
73972024-11-28T02:07:11.006ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
73982024-11-28T02:07:11.007ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "bfb0e163-953a-49fc-b031-d16afab1860d", "date": "Thu, 28 Nov 2024 02:07:10 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
73992024-11-28T02:07:11.007ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
74002024-11-28T02:07:11.009ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "5bdb70f6-862e-43af-a31f-871f27e07b4a", "date": "Thu, 28 Nov 2024 02:07:10 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
74012024-11-28T02:07:11.009ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
74022024-11-28T02:07:11.011ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "e507aaf2-2bcf-45f4-a0e8-7d0b39463331", "date": "Thu, 28 Nov 2024 02:07:10 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
74032024-11-28T02:07:11.011ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
74042024-11-28T02:07:11.012ZDEBGnexus (BackgroundTasks): client request SledAgent = 313847b3-d670-4167-9b09-9e7d3dfcb2a4 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
74052024-11-28T02:07:11.013ZDEBGnexus (BackgroundTasks): client response SledAgent = 313847b3-d670-4167-9b09-9e7d3dfcb2a4 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "549e4201-9707-4beb-9df0-fe7515207059", "content-length": "691", "date": "Thu, 28 Nov 2024 02:07:11 GMT"} })
74062024-11-28T02:07:11.013ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
74072024-11-28T02:07:11.013ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
74082024-11-28T02:07:11.013ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
74092024-11-28T02:07:11.014ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
74102024-11-28T02:07:11.014ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 534.158119ms iteration = 17 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
74112024-11-28T02:07:11.015ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 535.001672ms iteration = 30 status = {}
74122024-11-28T02:07:11.049ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
74132024-11-28T02:07:11.049ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
74142024-11-28T02:07:11.052ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 4bb2986f-2e92-4ac3-9dc1-591b6bb60b3e, rack_id: 92a8ff6e-21d3-4c28-bcad-e323847985dc, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(ea300318-9d62-42ed-80ea-21c86590a166) } rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc switch_location = Switch0
74152024-11-28T02:07:11.052ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
74162024-11-28T02:07:11.052ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
74172024-11-28T02:07:11.061ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
74182024-11-28T02:07:11.061ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "13f8148f-868a-4f7a-9f34-2ccd0d860d2b", "content-length": "134", "date": "Thu, 28 Nov 2024 02:07:10 GMT"} })
74192024-11-28T02:07:11.061ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1746 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
74202024-11-28T02:07:11.061ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1773 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = ea300318-9d62-42ed-80ea-21c86590a166
74212024-11-28T02:07:11.061ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
74222024-11-28T02:07:11.061ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
74232024-11-28T02:07:11.061ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
74242024-11-28T02:07:11.061ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
74252024-11-28T02:07:11.061ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
74262024-11-28T02:07:11.061ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1920 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
74272024-11-28T02:07:11.062ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
74282024-11-28T02:07:11.062ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
74292024-11-28T02:07:11.062ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc
74302024-11-28T02:07:11.062ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
74312024-11-28T02:07:11.062ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
74322024-11-28T02:07:11.064ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
74332024-11-28T02:07:11.064ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "dd5dce5d-e216-4ff9-a76e-bca3029736d6", "content-length": "2", "date": "Thu, 28 Nov 2024 02:07:10 GMT"} })
74342024-11-28T02:07:11.064ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
74352024-11-28T02:07:11.065ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "7444754e-c396-4a74-8ff6-c2d697700f74", "content-length": "52", "date": "Thu, 28 Nov 2024 02:07:10 GMT"} })
74362024-11-28T02:07:11.065ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc switch_location = Switch0
74372024-11-28T02:07:11.065ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
74382024-11-28T02:07:11.166ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "5b12f471-0114-4323-910d-e9ecaf1a348b", "date": "Thu, 28 Nov 2024 02:07:11 GMT"} })
74392024-11-28T02:07:11.166ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc switch = Switch0
74402024-11-28T02:07:11.166ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
74412024-11-28T02:07:11.167ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "7e36a452-5953-410b-b6df-2412b6987283", "content-length": "682", "date": "Thu, 28 Nov 2024 02:07:11 GMT"} })
74422024-11-28T02:07:11.167ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
74432024-11-28T02:07:11.167ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
74442024-11-28T02:07:11.171ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
74452024-11-28T02:07:11.171ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
74462024-11-28T02:07:11.175ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
74472024-11-28T02:07:11.176ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
74482024-11-28T02:07:11.178ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
74492024-11-28T02:07:11.178ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
74502024-11-28T02:07:11.185ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 705.876241ms iteration = 29 status = {}
74512024-11-28T02:07:11.850ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 03d5bd0e-97bb-4419-b53e-eed4c76acbfe roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
74522024-11-28T02:07:11.851ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 03d5bd0e-97bb-4419-b53e-eed4c76acbfe resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
74532024-11-28T02:07:11.852ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2157 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 03d5bd0e-97bb-4419-b53e-eed4c76acbfe response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
74542024-11-28T02:07:12.871ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 93328e60-b549-4017-b086-7cf2740d1a63 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
74552024-11-28T02:07:12.871ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 93328e60-b549-4017-b086-7cf2740d1a63 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
74562024-11-28T02:07:12.873ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2220 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 93328e60-b549-4017-b086-7cf2740d1a63 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
74572024-11-28T02:07:13.892ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 3bcec66f-2ac9-47e2-850e-f813fdbc9d59 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
74582024-11-28T02:07:13.893ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 3bcec66f-2ac9-47e2-850e-f813fdbc9d59 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
74592024-11-28T02:07:13.895ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2301 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 3bcec66f-2ac9-47e2-850e-f813fdbc9d59 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
74602024-11-28T02:07:14.194ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1481 local_addr = [fd00:1122:3344:101::b]:55621 method = GET remote_addr = [fd00:1122:3344:101::d]:56417 req_id = 797620ad-36e7-4308-8f09-d680f90e7f5d response_code = 200 uri = /2469a748-bcd2-4e0d-99cc-f79def0452cc
74612024-11-28T02:07:14.914ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = df6c9004-bf32-4f62-afb4-27dd96b0b06a roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
74622024-11-28T02:07:14.915ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = df6c9004-bf32-4f62-afb4-27dd96b0b06a resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
74632024-11-28T02:07:14.917ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2629 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = df6c9004-bf32-4f62-afb4-27dd96b0b06a response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
74642024-11-28T02:07:15.936ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = b50a3c92-584a-45f4-996e-457ed9f916bf roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
74652024-11-28T02:07:15.936ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = b50a3c92-584a-45f4-996e-457ed9f916bf resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
74662024-11-28T02:07:15.938ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2653 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = b50a3c92-584a-45f4-996e-457ed9f916bf response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
74672024-11-28T02:07:16.958ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 17107165-3fd1-4f3d-8369-198628a976e4 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
74682024-11-28T02:07:16.958ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 17107165-3fd1-4f3d-8369-198628a976e4 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
74692024-11-28T02:07:16.960ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2718 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 17107165-3fd1-4f3d-8369-198628a976e4 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
74702024-11-28T02:07:17.979ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = fa9148bb-d91d-4f12-b80f-1e09ff98f8bc roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
74712024-11-28T02:07:17.979ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = fa9148bb-d91d-4f12-b80f-1e09ff98f8bc resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
74722024-11-28T02:07:17.981ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2594 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = fa9148bb-d91d-4f12-b80f-1e09ff98f8bc response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
74732024-11-28T02:07:19.002ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 2c13f3a3-346a-4419-8612-4255fbd66fcb roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
74742024-11-28T02:07:19.002ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 2c13f3a3-346a-4419-8612-4255fbd66fcb resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
74752024-11-28T02:07:19.004ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2129 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 2c13f3a3-346a-4419-8612-4255fbd66fcb response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
74762024-11-28T02:07:20.024ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = b3b8a3c9-51c5-4b2a-a485-e2889dfebc44 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
74772024-11-28T02:07:20.024ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = b3b8a3c9-51c5-4b2a-a485-e2889dfebc44 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
74782024-11-28T02:07:20.026ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2718 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = b3b8a3c9-51c5-4b2a-a485-e2889dfebc44 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
74792024-11-28T02:07:20.479ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 86 reason = Timeout
74802024-11-28T02:07:20.479ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 86 reason = Timeout
74812024-11-28T02:07:20.479ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
74822024-11-28T02:07:20.479ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
74832024-11-28T02:07:20.480ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
74842024-11-28T02:07:20.480ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
74852024-11-28T02:07:20.482ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
74862024-11-28T02:07:20.482ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
74872024-11-28T02:07:20.482ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
74882024-11-28T02:07:20.484ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 4.506258ms iteration = 86 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
74892024-11-28T02:07:20.485ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
74902024-11-28T02:07:20.485ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
74912024-11-28T02:07:20.485ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
74922024-11-28T02:07:20.486ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
74932024-11-28T02:07:20.487ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
74942024-11-28T02:07:20.488ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
74952024-11-28T02:07:20.491ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 67ac247f-c934-4c78-857a-fba4fbeb9937, lookup_type: ById(67ac247f-c934-4c78-857a-fba4fbeb9937) } result = Ok(())
74962024-11-28T02:07:20.491ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
74972024-11-28T02:07:20.491ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
74982024-11-28T02:07:20.500ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 67ac247f-c934-4c78-857a-fba4fbeb9937 original_time_created = 2024-11-28 01:53:03.159475 UTC target_id = 67ac247f-c934-4c78-857a-fba4fbeb9937 time_created = 2024-11-28 01:53:03.159475 UTC
74992024-11-28T02:07:20.500ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 20.69824ms iteration = 86 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"67ac247f-c934-4c78-857a-fba4fbeb9937","time_created":"2024-11-28T01:53:03.159475Z"}
75002024-11-28T02:07:21.045ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 0425e471-be33-4c98-9562-f5a260798e73 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
75012024-11-28T02:07:21.046ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 0425e471-be33-4c98-9562-f5a260798e73 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
75022024-11-28T02:07:21.047ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1971 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 0425e471-be33-4c98-9562-f5a260798e73 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
75032024-11-28T02:07:22.067ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 76aeedf9-9cf3-4bfe-b77f-5da932cc5af1 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
75042024-11-28T02:07:22.067ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 76aeedf9-9cf3-4bfe-b77f-5da932cc5af1 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
75052024-11-28T02:07:22.069ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2251 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 76aeedf9-9cf3-4bfe-b77f-5da932cc5af1 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
75062024-11-28T02:07:23.088ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 18f4e883-3806-433d-8a94-31ed55d53a1a roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
75072024-11-28T02:07:23.089ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 18f4e883-3806-433d-8a94-31ed55d53a1a resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
75082024-11-28T02:07:23.090ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2139 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 18f4e883-3806-433d-8a94-31ed55d53a1a response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
75092024-11-28T02:07:24.110ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 85e8ebd6-0f9e-445d-8830-018c4ce4d931 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
75102024-11-28T02:07:24.110ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 85e8ebd6-0f9e-445d-8830-018c4ce4d931 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
75112024-11-28T02:07:24.112ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2436 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 85e8ebd6-0f9e-445d-8830-018c4ce4d931 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
75122024-11-28T02:07:24.195ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1280 local_addr = [fd00:1122:3344:101::b]:55621 method = GET remote_addr = [fd00:1122:3344:101::d]:56417 req_id = 5d901417-ce50-4b3d-add3-169c59885e8a response_code = 200 uri = /2469a748-bcd2-4e0d-99cc-f79def0452cc
75132024-11-28T02:07:25.131ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 926d0d92-74a4-41e1-8e6b-4ff559825285 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
75142024-11-28T02:07:25.131ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 926d0d92-74a4-41e1-8e6b-4ff559825285 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
75152024-11-28T02:07:25.134ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2952 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 926d0d92-74a4-41e1-8e6b-4ff559825285 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
75162024-11-28T02:07:26.153ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 54cb5625-0adb-457c-94f0-333a2563d6ae roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
75172024-11-28T02:07:26.153ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 54cb5625-0adb-457c-94f0-333a2563d6ae resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
75182024-11-28T02:07:26.155ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2515 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 54cb5625-0adb-457c-94f0-333a2563d6ae response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
75192024-11-28T02:07:27.175ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 5426605a-ca3c-4b72-9523-4903a3d5b498 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
75202024-11-28T02:07:27.175ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 5426605a-ca3c-4b72-9523-4903a3d5b498 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
75212024-11-28T02:07:27.177ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2762 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 5426605a-ca3c-4b72-9523-4903a3d5b498 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
75222024-11-28T02:07:28.197ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 80e0918d-1d25-4561-91e2-713a4823e366 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
75232024-11-28T02:07:28.198ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 80e0918d-1d25-4561-91e2-713a4823e366 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
75242024-11-28T02:07:28.199ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1992 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 80e0918d-1d25-4561-91e2-713a4823e366 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
75252024-11-28T02:07:29.218ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = ff968291-3f98-439c-899d-b6076af8fee4 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
75262024-11-28T02:07:29.219ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = ff968291-3f98-439c-899d-b6076af8fee4 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
75272024-11-28T02:07:29.221ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2853 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = ff968291-3f98-439c-899d-b6076af8fee4 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
75282024-11-28T02:07:30.239ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 835b7c58-8831-433c-8877-39f2726514ba roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
75292024-11-28T02:07:30.240ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 835b7c58-8831-433c-8877-39f2726514ba resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
75302024-11-28T02:07:30.241ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2345 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 835b7c58-8831-433c-8877-39f2726514ba response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
75312024-11-28T02:07:30.479ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 87 reason = Timeout
75322024-11-28T02:07:30.479ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 87 reason = Timeout
75332024-11-28T02:07:30.479ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
75342024-11-28T02:07:30.479ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
75352024-11-28T02:07:30.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
75362024-11-28T02:07:30.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
75372024-11-28T02:07:30.481ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
75382024-11-28T02:07:30.481ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
75392024-11-28T02:07:30.482ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
75402024-11-28T02:07:30.484ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
75412024-11-28T02:07:30.484ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
75422024-11-28T02:07:30.484ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
75432024-11-28T02:07:30.485ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
75442024-11-28T02:07:30.486ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
75452024-11-28T02:07:30.487ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
75462024-11-28T02:07:30.490ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 67ac247f-c934-4c78-857a-fba4fbeb9937, lookup_type: ById(67ac247f-c934-4c78-857a-fba4fbeb9937) } result = Ok(())
75472024-11-28T02:07:30.490ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
75482024-11-28T02:07:30.490ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
75492024-11-28T02:07:30.500ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 67ac247f-c934-4c78-857a-fba4fbeb9937 original_time_created = 2024-11-28 01:53:03.159475 UTC target_id = 67ac247f-c934-4c78-857a-fba4fbeb9937 time_created = 2024-11-28 01:53:03.159475 UTC
75502024-11-28T02:07:30.500ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 21.752008ms iteration = 87 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"67ac247f-c934-4c78-857a-fba4fbeb9937","time_created":"2024-11-28T01:53:03.159475Z"}
75512024-11-28T02:07:30.510ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 31.396475ms iteration = 87 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
75522024-11-28T02:07:31.261ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 0d8f6fda-d513-4b9c-8654-146e90a6c5b6 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
75532024-11-28T02:07:31.261ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 0d8f6fda-d513-4b9c-8654-146e90a6c5b6 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
75542024-11-28T02:07:31.263ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2514 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 0d8f6fda-d513-4b9c-8654-146e90a6c5b6 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
75552024-11-28T02:07:32.283ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = b2110fea-7cae-479f-8aee-d29cbfa60595 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
75562024-11-28T02:07:32.283ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = b2110fea-7cae-479f-8aee-d29cbfa60595 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
75572024-11-28T02:07:32.285ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2447 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = b2110fea-7cae-479f-8aee-d29cbfa60595 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
75582024-11-28T02:07:33.304ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 00e76957-b25d-4100-8f30-c061ffafd46e roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
75592024-11-28T02:07:33.304ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 00e76957-b25d-4100-8f30-c061ffafd46e resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
75602024-11-28T02:07:33.306ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2378 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 00e76957-b25d-4100-8f30-c061ffafd46e response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
75612024-11-28T02:07:34.196ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1810 local_addr = [fd00:1122:3344:101::b]:55621 method = GET remote_addr = [fd00:1122:3344:101::d]:56417 req_id = a5acfa15-6c4e-4be8-aadc-246b705d6c26 response_code = 200 uri = /2469a748-bcd2-4e0d-99cc-f79def0452cc
75622024-11-28T02:07:34.324ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = ce8e04ec-fdc8-4056-8dbb-d1387bdd857f roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
75632024-11-28T02:07:34.324ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = ce8e04ec-fdc8-4056-8dbb-d1387bdd857f resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
75642024-11-28T02:07:34.326ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2477 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = ce8e04ec-fdc8-4056-8dbb-d1387bdd857f response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
75652024-11-28T02:07:35.345ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = a7e9c2ad-f924-414a-889a-87fc284d4d8b roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
75662024-11-28T02:07:35.345ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = a7e9c2ad-f924-414a-889a-87fc284d4d8b resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
75672024-11-28T02:07:35.347ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2355 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = a7e9c2ad-f924-414a-889a-87fc284d4d8b response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
75682024-11-28T02:07:36.366ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 7c587ccd-bf71-4c54-a59a-cab01755b154 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
75692024-11-28T02:07:36.367ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 7c587ccd-bf71-4c54-a59a-cab01755b154 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
75702024-11-28T02:07:36.369ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2494 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 7c587ccd-bf71-4c54-a59a-cab01755b154 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
75712024-11-28T02:07:37.388ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 719b9104-4892-4939-a7e1-20734e6944dc roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
75722024-11-28T02:07:37.388ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 719b9104-4892-4939-a7e1-20734e6944dc resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
75732024-11-28T02:07:37.389ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1866 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 719b9104-4892-4939-a7e1-20734e6944dc response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
75742024-11-28T02:07:38.409ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 0e4e5876-5510-46e9-95ec-edddbd34b1d7 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
75752024-11-28T02:07:38.409ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 0e4e5876-5510-46e9-95ec-edddbd34b1d7 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
75762024-11-28T02:07:38.412ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2427 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 0e4e5876-5510-46e9-95ec-edddbd34b1d7 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
75772024-11-28T02:07:38.679ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:40402 req_id = 333941f2-326f-4eeb-afe4-7573e2f638dd roles = RoleSet { roles: {} } uri = /nat/ipv4/changeset/10?limit=100
75782024-11-28T02:07:38.679ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:40402 req_id = 333941f2-326f-4eeb-afe4-7573e2f638dd resource = Database result = Ok(()) uri = /nat/ipv4/changeset/10?limit=100
75792024-11-28T02:07:38.682ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 3168 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:40402 req_id = 333941f2-326f-4eeb-afe4-7573e2f638dd response_code = 200 uri = /nat/ipv4/changeset/10?limit=100
75802024-11-28T02:07:39.430ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 60300972-c697-4ce2-b5b5-426ac10478e7 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
75812024-11-28T02:07:39.431ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 60300972-c697-4ce2-b5b5-426ac10478e7 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
75822024-11-28T02:07:39.433ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2362 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 60300972-c697-4ce2-b5b5-426ac10478e7 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
75832024-11-28T02:07:40.451ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 90ec6ebd-e174-4077-b5e0-6c4213daf48f roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
75842024-11-28T02:07:40.452ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 90ec6ebd-e174-4077-b5e0-6c4213daf48f resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
75852024-11-28T02:07:40.454ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 3263 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 90ec6ebd-e174-4077-b5e0-6c4213daf48f response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
75862024-11-28T02:07:40.479ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 30 reason = Timeout
75872024-11-28T02:07:40.479ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 30 reason = Timeout
75882024-11-28T02:07:40.479ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
75892024-11-28T02:07:40.479ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
75902024-11-28T02:07:40.479ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 30 reason = Timeout
75912024-11-28T02:07:40.479ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 88 reason = Timeout
75922024-11-28T02:07:40.479ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
75932024-11-28T02:07:40.479ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
75942024-11-28T02:07:40.479ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 30 reason = Timeout
75952024-11-28T02:07:40.479ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 30 reason = Timeout
75962024-11-28T02:07:40.479ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
75972024-11-28T02:07:40.480ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
75982024-11-28T02:07:40.480ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 34 reason = Timeout
75992024-11-28T02:07:40.480ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
76002024-11-28T02:07:40.480ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 88 reason = Timeout
76012024-11-28T02:07:40.480ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 30 reason = Timeout
76022024-11-28T02:07:40.480ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
76032024-11-28T02:07:40.480ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
76042024-11-28T02:07:40.480ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 30 reason = Timeout
76052024-11-28T02:07:40.480ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 30 reason = Timeout
76062024-11-28T02:07:40.480ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
76072024-11-28T02:07:40.480ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
76082024-11-28T02:07:40.480ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 30 reason = Timeout
76092024-11-28T02:07:40.480ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
76102024-11-28T02:07:40.480ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 30 reason = Timeout
76112024-11-28T02:07:40.480ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 31 reason = Timeout
76122024-11-28T02:07:40.480ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
76132024-11-28T02:07:40.480ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 30 reason = Timeout
76142024-11-28T02:07:40.480ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 32 reason = Timeout
76152024-11-28T02:07:40.480ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
76162024-11-28T02:07:40.480ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
76172024-11-28T02:07:40.480ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
76182024-11-28T02:07:40.480ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 30 reason = Timeout
76192024-11-28T02:07:40.480ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
76202024-11-28T02:07:40.480ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
76212024-11-28T02:07:40.480ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
76222024-11-28T02:07:40.480ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
76232024-11-28T02:07:40.480ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
76242024-11-28T02:07:40.480ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
76252024-11-28T02:07:40.480ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
76262024-11-28T02:07:40.480ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
76272024-11-28T02:07:40.480ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
76282024-11-28T02:07:40.480ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
76292024-11-28T02:07:40.480ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
76302024-11-28T02:07:40.480ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
76312024-11-28T02:07:40.480ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
76322024-11-28T02:07:40.480ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
76332024-11-28T02:07:40.480ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
76342024-11-28T02:07:40.480ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
76352024-11-28T02:07:40.480ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
76362024-11-28T02:07:40.480ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
76372024-11-28T02:07:40.482ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
76382024-11-28T02:07:40.483ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
76392024-11-28T02:07:40.483ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
76402024-11-28T02:07:40.483ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
76412024-11-28T02:07:40.483ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
76422024-11-28T02:07:40.484ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 4.284954ms iteration = 30 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
76432024-11-28T02:07:40.485ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
76442024-11-28T02:07:40.485ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
76452024-11-28T02:07:40.485ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
76462024-11-28T02:07:40.486ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
76472024-11-28T02:07:40.486ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
76482024-11-28T02:07:40.486ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
76492024-11-28T02:07:40.486ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
76502024-11-28T02:07:40.486ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
76512024-11-28T02:07:40.486ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
76522024-11-28T02:07:40.491ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-313847b3-d670-4167-9b09-9e7d3dfcb2a4.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-313847b3-d670-4167-9b09-9e7d3dfcb2a4.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1726, tv_nsec: 960931576 } })
76532024-11-28T02:07:40.491ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1103
76542024-11-28T02:07:40.493ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 13.114927ms iteration = 30 status = {"errors":[],"garbage_collect_requested":[]}
76552024-11-28T02:07:40.497ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-313847b3-d670-4167-9b09-9e7d3dfcb2a4.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-313847b3-d670-4167-9b09-9e7d3dfcb2a4.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1726, tv_nsec: 967602222 } })
76562024-11-28T02:07:40.498ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1103
76572024-11-28T02:07:40.499ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
76582024-11-28T02:07:40.499ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 19.683832ms iteration = 88 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
76592024-11-28T02:07:40.499ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
76602024-11-28T02:07:40.501ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
76612024-11-28T02:07:40.501ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
76622024-11-28T02:07:40.501ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
76632024-11-28T02:07:40.502ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-313847b3-d670-4167-9b09-9e7d3dfcb2a4.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-313847b3-d670-4167-9b09-9e7d3dfcb2a4.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1726, tv_nsec: 971811875 } })
76642024-11-28T02:07:40.502ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1103 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc
76652024-11-28T02:07:40.503ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
76662024-11-28T02:07:40.503ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
76672024-11-28T02:07:40.503ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
76682024-11-28T02:07:40.505ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
76692024-11-28T02:07:40.505ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
76702024-11-28T02:07:40.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
76712024-11-28T02:07:40.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
76722024-11-28T02:07:40.508ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
76732024-11-28T02:07:40.510ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
76742024-11-28T02:07:40.510ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
76752024-11-28T02:07:40.510ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
76762024-11-28T02:07:40.528ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
76772024-11-28T02:07:40.529ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
76782024-11-28T02:07:40.529ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
76792024-11-28T02:07:40.529ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
76802024-11-28T02:07:40.531ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 51.396002ms iteration = 30 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
76812024-11-28T02:07:40.533ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
76822024-11-28T02:07:40.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
76832024-11-28T02:07:40.536ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
76842024-11-28T02:07:40.538ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
76852024-11-28T02:07:40.539ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 59.248269ms iteration = 30 status = {"errors":[],"records_set_to_done":[]}
76862024-11-28T02:07:40.539ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
76872024-11-28T02:07:40.539ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
76882024-11-28T02:07:40.539ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
76892024-11-28T02:07:40.539ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
76902024-11-28T02:07:40.540ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
76912024-11-28T02:07:40.540ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
76922024-11-28T02:07:40.540ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
76932024-11-28T02:07:40.541ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
76942024-11-28T02:07:40.541ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
76952024-11-28T02:07:40.541ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
76962024-11-28T02:07:40.542ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
76972024-11-28T02:07:40.542ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
76982024-11-28T02:07:40.544ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
76992024-11-28T02:07:40.544ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
77002024-11-28T02:07:40.544ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
77012024-11-28T02:07:40.544ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
77022024-11-28T02:07:40.545ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
77032024-11-28T02:07:40.548ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 67ac247f-c934-4c78-857a-fba4fbeb9937, lookup_type: ById(67ac247f-c934-4c78-857a-fba4fbeb9937) } result = Ok(())
77042024-11-28T02:07:40.548ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
77052024-11-28T02:07:40.548ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
77062024-11-28T02:07:40.548ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
77072024-11-28T02:07:40.548ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 68.831715ms iteration = 30 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
77082024-11-28T02:07:40.561ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
77092024-11-28T02:07:40.561ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
77102024-11-28T02:07:40.561ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "3fadbe12-1299-46bb-9275-f3bbdcc384a3", "content-length": "26", "date": "Thu, 28 Nov 2024 02:07:40 GMT"} })
77112024-11-28T02:07:40.561ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
77122024-11-28T02:07:40.563ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
77132024-11-28T02:07:40.564ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
77142024-11-28T02:07:40.564ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
77152024-11-28T02:07:40.572ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
77162024-11-28T02:07:40.572ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
77172024-11-28T02:07:40.572ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
77182024-11-28T02:07:40.573ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
77192024-11-28T02:07:40.574ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 94.338539ms iteration = 30 status = {"errors":[],"requests_created_ok":[],"start_invoked_ok":[]}
77202024-11-28T02:07:40.601ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
77212024-11-28T02:07:40.601ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
77222024-11-28T02:07:40.601ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "8bef00fd-fc6b-4dea-b30f-6ec51bbefec6", "content-length": "26", "date": "Thu, 28 Nov 2024 02:07:40 GMT"} })
77232024-11-28T02:07:40.601ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
77242024-11-28T02:07:40.605ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 125.469449ms iteration = 34 status = {"physical_disks_added":0}
77252024-11-28T02:07:40.607ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
77262024-11-28T02:07:40.609ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1111 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc zone_address = fd00:1122:3344:101::2
77272024-11-28T02:07:40.609ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
77282024-11-28T02:07:40.609ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "5f55b01b-6f71-4d17-91f8-35903cefa68a", "content-length": "26", "date": "Thu, 28 Nov 2024 02:07:40 GMT"} })
77292024-11-28T02:07:40.609ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1116 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
77302024-11-28T02:07:40.611ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc
77312024-11-28T02:07:40.623ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
77322024-11-28T02:07:40.623ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
77332024-11-28T02:07:40.655ZINFOnexus (BackgroundTasks): VPC route manager sled 313847b3-d670-4167-9b09-9e7d3dfcb2a4 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
77342024-11-28T02:07:40.655ZDEBGnexus (BackgroundTasks): client request SledAgent = 313847b3-d670-4167-9b09-9e7d3dfcb2a4 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
77352024-11-28T02:07:40.655ZDEBGnexus (BackgroundTasks): client response SledAgent = 313847b3-d670-4167-9b09-9e7d3dfcb2a4 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "28a030be-95e3-486f-98c8-42e22fd0db20", "content-length": "372", "date": "Thu, 28 Nov 2024 02:07:40 GMT"} })
77362024-11-28T02:07:40.655ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
77372024-11-28T02:07:40.656ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
77382024-11-28T02:07:40.657ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
77392024-11-28T02:07:40.658ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
77402024-11-28T02:07:40.658ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
77412024-11-28T02:07:40.658ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 178.761845ms iteration = 30 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
77422024-11-28T02:07:40.660ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 313847b3-d670-4167-9b09-9e7d3dfcb2a4, lookup_type: ById(313847b3-d670-4167-9b09-9e7d3dfcb2a4) } result = Ok(())
77432024-11-28T02:07:40.660ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
77442024-11-28T02:07:40.660ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
77452024-11-28T02:07:40.661ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 181.698127ms iteration = 30 status = 0
77462024-11-28T02:07:40.665ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
77472024-11-28T02:07:40.665ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
77482024-11-28T02:07:40.667ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
77492024-11-28T02:07:40.668ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 67ac247f-c934-4c78-857a-fba4fbeb9937 original_time_created = 2024-11-28 01:53:03.159475 UTC target_id = 67ac247f-c934-4c78-857a-fba4fbeb9937 time_created = 2024-11-28 01:53:03.159475 UTC
77502024-11-28T02:07:40.668ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 188.661817ms iteration = 88 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"67ac247f-c934-4c78-857a-fba4fbeb9937","time_created":"2024-11-28T01:53:03.159475Z"}
77512024-11-28T02:07:40.669ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
77522024-11-28T02:07:40.669ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
77532024-11-28T02:07:40.669ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
77542024-11-28T02:07:40.671ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
77552024-11-28T02:07:40.673ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
77562024-11-28T02:07:40.673ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
77572024-11-28T02:07:40.673ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
77582024-11-28T02:07:40.675ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
77592024-11-28T02:07:40.675ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "3b252439-9e36-4895-afe1-dcdff4aa6d19", "content-length": "2", "date": "Thu, 28 Nov 2024 02:07:40 GMT"} })
77602024-11-28T02:07:40.682ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
77612024-11-28T02:07:40.682ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
77622024-11-28T02:07:40.684ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 204.715195ms iteration = 30 status = null
77632024-11-28T02:07:40.693ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 213.027601ms iteration = 30 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[]}
77642024-11-28T02:07:40.694ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 214.186061ms iteration = 30 status = {}
77652024-11-28T02:07:40.723ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
77662024-11-28T02:07:40.723ZDEBGnexus (BackgroundTasks): client request SledAgent = 313847b3-d670-4167-9b09-9e7d3dfcb2a4 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
77672024-11-28T02:07:40.724ZDEBGnexus (BackgroundTasks): client response SledAgent = 313847b3-d670-4167-9b09-9e7d3dfcb2a4 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "69295aae-ea4f-420d-a68f-c5a734a71f04", "date": "Thu, 28 Nov 2024 02:07:40 GMT"} })
77682024-11-28T02:07:40.724ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
77692024-11-28T02:07:40.724ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
77702024-11-28T02:07:40.727ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
77712024-11-28T02:07:40.727ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
77722024-11-28T02:07:40.731ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
77732024-11-28T02:07:40.731ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
77742024-11-28T02:07:40.736ZINFOnexus (BackgroundTasks): VPC route manager sled 313847b3-d670-4167-9b09-9e7d3dfcb2a4 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
77752024-11-28T02:07:40.736ZDEBGnexus (BackgroundTasks): client request SledAgent = 313847b3-d670-4167-9b09-9e7d3dfcb2a4 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
77762024-11-28T02:07:40.737ZDEBGnexus (BackgroundTasks): client response SledAgent = 313847b3-d670-4167-9b09-9e7d3dfcb2a4 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "5542a26e-eb72-4a0c-bc5a-38e2a05f5d0c", "date": "Thu, 28 Nov 2024 02:07:40 GMT"} })
77772024-11-28T02:07:40.740ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 260.77974ms iteration = 32 status = {}
77782024-11-28T02:07:40.804ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
77792024-11-28T02:07:40.804ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
77802024-11-28T02:07:40.815ZDEBGnexus (BackgroundTasks): client request SledAgent = 313847b3-d670-4167-9b09-9e7d3dfcb2a4 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
77812024-11-28T02:07:40.818ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 4bb2986f-2e92-4ac3-9dc1-591b6bb60b3e, rack_id: 92a8ff6e-21d3-4c28-bcad-e323847985dc, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(ea300318-9d62-42ed-80ea-21c86590a166) } rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc switch_location = Switch0
77822024-11-28T02:07:40.818ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
77832024-11-28T02:07:40.818ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
77842024-11-28T02:07:40.826ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
77852024-11-28T02:07:40.826ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "d397d41d-a02f-4978-93df-85bb6c188d9a", "content-length": "134", "date": "Thu, 28 Nov 2024 02:07:40 GMT"} })
77862024-11-28T02:07:40.826ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1746 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
77872024-11-28T02:07:40.826ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1773 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = ea300318-9d62-42ed-80ea-21c86590a166
77882024-11-28T02:07:40.826ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
77892024-11-28T02:07:40.826ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
77902024-11-28T02:07:40.826ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
77912024-11-28T02:07:40.826ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
77922024-11-28T02:07:40.826ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
77932024-11-28T02:07:40.826ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1920 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
77942024-11-28T02:07:40.827ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
77952024-11-28T02:07:40.827ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
77962024-11-28T02:07:40.827ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc
77972024-11-28T02:07:40.827ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
77982024-11-28T02:07:40.827ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
77992024-11-28T02:07:40.829ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
78002024-11-28T02:07:40.829ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "0e15e985-8e5f-4a7e-b6f9-06fe6d2b04fc", "content-length": "2", "date": "Thu, 28 Nov 2024 02:07:40 GMT"} })
78012024-11-28T02:07:40.829ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
78022024-11-28T02:07:40.829ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "98ef1e39-6011-429c-9268-d48d934df4dd", "content-length": "52", "date": "Thu, 28 Nov 2024 02:07:40 GMT"} })
78032024-11-28T02:07:40.829ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc switch_location = Switch0
78042024-11-28T02:07:40.829ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
78052024-11-28T02:07:40.877ZDEBGnexus (BackgroundTasks): client response SledAgent = 313847b3-d670-4167-9b09-9e7d3dfcb2a4 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "e997b14d-8cc4-4292-afdb-bce1f2da5d5b", "content-length": "691", "date": "Thu, 28 Nov 2024 02:07:40 GMT"} })
78062024-11-28T02:07:40.877ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
78072024-11-28T02:07:40.877ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
78082024-11-28T02:07:40.877ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
78092024-11-28T02:07:40.879ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 399.278945ms iteration = 31 status = {}
78102024-11-28T02:07:40.965ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "9556f4f6-1ad4-4799-85bf-e7b4c6bd4da0", "date": "Thu, 28 Nov 2024 02:07:40 GMT"} })
78112024-11-28T02:07:40.965ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc switch = Switch0
78122024-11-28T02:07:40.965ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
78132024-11-28T02:07:40.965ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "c0c20c12-455b-4ae5-a760-d01aac624053", "content-length": "682", "date": "Thu, 28 Nov 2024 02:07:40 GMT"} })
78142024-11-28T02:07:40.965ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
78152024-11-28T02:07:40.966ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
78162024-11-28T02:07:40.969ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
78172024-11-28T02:07:40.969ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
78182024-11-28T02:07:40.972ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
78192024-11-28T02:07:40.972ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
78202024-11-28T02:07:40.973ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
78212024-11-28T02:07:40.974ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
78222024-11-28T02:07:40.980ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 500.973352ms iteration = 30 status = {}
78232024-11-28T02:07:41.477ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 139b2e94-5b65-4ece-bf36-21aed54fe364 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
78242024-11-28T02:07:41.477ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 139b2e94-5b65-4ece-bf36-21aed54fe364 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
78252024-11-28T02:07:41.479ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2388 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 139b2e94-5b65-4ece-bf36-21aed54fe364 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
78262024-11-28T02:07:42.498ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 4cc85671-5070-434f-a703-a29bf297a0f1 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
78272024-11-28T02:07:42.499ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 4cc85671-5070-434f-a703-a29bf297a0f1 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
78282024-11-28T02:07:42.501ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2893 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 4cc85671-5070-434f-a703-a29bf297a0f1 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
78292024-11-28T02:07:43.520ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 8ab5b96f-21b1-469f-9a4a-248819fb9de8 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
78302024-11-28T02:07:43.521ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 8ab5b96f-21b1-469f-9a4a-248819fb9de8 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
78312024-11-28T02:07:43.522ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2167 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 8ab5b96f-21b1-469f-9a4a-248819fb9de8 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
78322024-11-28T02:07:44.195ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1568 local_addr = [fd00:1122:3344:101::b]:55621 method = GET remote_addr = [fd00:1122:3344:101::d]:56417 req_id = c514822a-4241-48eb-8739-e768cdefb73e response_code = 200 uri = /2469a748-bcd2-4e0d-99cc-f79def0452cc
78332024-11-28T02:07:44.543ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 8f7bf3fb-5f2d-4c0e-80fe-eeb2403eb467 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
78342024-11-28T02:07:44.543ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 8f7bf3fb-5f2d-4c0e-80fe-eeb2403eb467 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
78352024-11-28T02:07:44.546ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2824 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 8f7bf3fb-5f2d-4c0e-80fe-eeb2403eb467 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
78362024-11-28T02:07:45.566ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = db1838fc-70a7-4a8c-a9fe-be8381c28b0e roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
78372024-11-28T02:07:45.566ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = db1838fc-70a7-4a8c-a9fe-be8381c28b0e resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
78382024-11-28T02:07:45.568ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2483 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = db1838fc-70a7-4a8c-a9fe-be8381c28b0e response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
78392024-11-28T02:07:46.588ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 539515f4-671d-48e1-baab-d6dbc1359490 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
78402024-11-28T02:07:46.588ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 539515f4-671d-48e1-baab-d6dbc1359490 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
78412024-11-28T02:07:46.590ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2668 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 539515f4-671d-48e1-baab-d6dbc1359490 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
78422024-11-28T02:07:47.609ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 4b6a9d29-5a77-4efb-918e-28955f7d68ab roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
78432024-11-28T02:07:47.610ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 4b6a9d29-5a77-4efb-918e-28955f7d68ab resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
78442024-11-28T02:07:47.612ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2752 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 4b6a9d29-5a77-4efb-918e-28955f7d68ab response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
78452024-11-28T02:07:48.631ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 6b6236fa-ab3d-4baf-bfff-1880c23e3499 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
78462024-11-28T02:07:48.632ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 6b6236fa-ab3d-4baf-bfff-1880c23e3499 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
78472024-11-28T02:07:48.633ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1881 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 6b6236fa-ab3d-4baf-bfff-1880c23e3499 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
78482024-11-28T02:07:49.652ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 01fb9813-e2b5-499e-91b3-2e66b42f8e22 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
78492024-11-28T02:07:49.653ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 01fb9813-e2b5-499e-91b3-2e66b42f8e22 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
78502024-11-28T02:07:49.655ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2392 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 01fb9813-e2b5-499e-91b3-2e66b42f8e22 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
78512024-11-28T02:07:50.479ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 89 reason = Timeout
78522024-11-28T02:07:50.479ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
78532024-11-28T02:07:50.479ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 89 reason = Timeout
78542024-11-28T02:07:50.479ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
78552024-11-28T02:07:50.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
78562024-11-28T02:07:50.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
78572024-11-28T02:07:50.480ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
78582024-11-28T02:07:50.481ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
78592024-11-28T02:07:50.481ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
78602024-11-28T02:07:50.482ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.568612ms iteration = 89 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
78612024-11-28T02:07:50.484ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
78622024-11-28T02:07:50.484ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
78632024-11-28T02:07:50.484ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
78642024-11-28T02:07:50.486ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
78652024-11-28T02:07:50.486ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
78662024-11-28T02:07:50.487ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
78672024-11-28T02:07:50.490ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 67ac247f-c934-4c78-857a-fba4fbeb9937, lookup_type: ById(67ac247f-c934-4c78-857a-fba4fbeb9937) } result = Ok(())
78682024-11-28T02:07:50.490ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
78692024-11-28T02:07:50.490ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
78702024-11-28T02:07:50.498ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 67ac247f-c934-4c78-857a-fba4fbeb9937 original_time_created = 2024-11-28 01:53:03.159475 UTC target_id = 67ac247f-c934-4c78-857a-fba4fbeb9937 time_created = 2024-11-28 01:53:03.159475 UTC
78712024-11-28T02:07:50.498ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 19.720493ms iteration = 89 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"67ac247f-c934-4c78-857a-fba4fbeb9937","time_created":"2024-11-28T01:53:03.159475Z"}
78722024-11-28T02:07:50.675ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = e27bb515-07a1-4d85-a4eb-16245253a3ed roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
78732024-11-28T02:07:50.675ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = e27bb515-07a1-4d85-a4eb-16245253a3ed resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
78742024-11-28T02:07:50.676ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1673 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = e27bb515-07a1-4d85-a4eb-16245253a3ed response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
78752024-11-28T02:07:51.696ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = da001c4a-5fdc-45d0-95a5-8c70d25d27b2 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
78762024-11-28T02:07:51.696ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = da001c4a-5fdc-45d0-95a5-8c70d25d27b2 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
78772024-11-28T02:07:51.698ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2197 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = da001c4a-5fdc-45d0-95a5-8c70d25d27b2 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
78782024-11-28T02:07:52.716ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 348da7ab-4423-44f7-b62a-c34994c3b8a8 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
78792024-11-28T02:07:52.717ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 348da7ab-4423-44f7-b62a-c34994c3b8a8 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
78802024-11-28T02:07:52.718ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2307 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 348da7ab-4423-44f7-b62a-c34994c3b8a8 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
78812024-11-28T02:07:53.736ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 44f5aa05-67a5-48c1-b2cf-50a1127defe2 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
78822024-11-28T02:07:53.737ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 44f5aa05-67a5-48c1-b2cf-50a1127defe2 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
78832024-11-28T02:07:53.739ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2717 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 44f5aa05-67a5-48c1-b2cf-50a1127defe2 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
78842024-11-28T02:07:54.186ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:55495 req_id = d0d0a77a-6dd3-4569-9a24-36a956bea904 roles = RoleSet { roles: {} } uri = /metrics/collectors/119eed64-2e24-4d5f-8c42-be6cd8bec8b5/producers?sort_by=id_ascending
78852024-11-28T02:07:54.187ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:55495 req_id = d0d0a77a-6dd3-4569-9a24-36a956bea904 resource = Database result = Ok(()) uri = /metrics/collectors/119eed64-2e24-4d5f-8c42-be6cd8bec8b5/producers?sort_by=id_ascending
78862024-11-28T02:07:54.190ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 4031 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:55495 req_id = d0d0a77a-6dd3-4569-9a24-36a956bea904 response_code = 200 uri = /metrics/collectors/119eed64-2e24-4d5f-8c42-be6cd8bec8b5/producers?sort_by=id_ascending
78872024-11-28T02:07:54.191ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:55495 req_id = dbbd141a-2faa-4f1e-8eb8-19a069a3555d roles = RoleSet { roles: {} } uri = /metrics/collectors/119eed64-2e24-4d5f-8c42-be6cd8bec8b5/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJjNDM0YjQ0OC01MTk5LTQ2NmUtYWYzMi00ZjYyOWUzMWM3YjEifX0%3D
78882024-11-28T02:07:54.191ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:55495 req_id = dbbd141a-2faa-4f1e-8eb8-19a069a3555d resource = Database result = Ok(()) uri = /metrics/collectors/119eed64-2e24-4d5f-8c42-be6cd8bec8b5/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJjNDM0YjQ0OC01MTk5LTQ2NmUtYWYzMi00ZjYyOWUzMWM3YjEifX0%3D
78892024-11-28T02:07:54.193ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2582 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:55495 req_id = dbbd141a-2faa-4f1e-8eb8-19a069a3555d response_code = 200 uri = /metrics/collectors/119eed64-2e24-4d5f-8c42-be6cd8bec8b5/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJjNDM0YjQ0OC01MTk5LTQ2NmUtYWYzMi00ZjYyOWUzMWM3YjEifX0%3D
78902024-11-28T02:07:54.195ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1360 local_addr = [fd00:1122:3344:101::b]:55621 method = GET remote_addr = [fd00:1122:3344:101::d]:56417 req_id = 916c697a-50aa-4200-bb68-2161423c642d response_code = 200 uri = /2469a748-bcd2-4e0d-99cc-f79def0452cc
78912024-11-28T02:07:54.756ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 24af0044-b8f7-4fab-8f2c-76333fb36dcc roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
78922024-11-28T02:07:54.757ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 24af0044-b8f7-4fab-8f2c-76333fb36dcc resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
78932024-11-28T02:07:54.759ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2665 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 24af0044-b8f7-4fab-8f2c-76333fb36dcc response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
78942024-11-28T02:07:55.780ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = fe2e9dd5-c01a-47a2-91aa-c9cc6e6def01 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
78952024-11-28T02:07:55.780ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = fe2e9dd5-c01a-47a2-91aa-c9cc6e6def01 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
78962024-11-28T02:07:55.782ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2230 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = fe2e9dd5-c01a-47a2-91aa-c9cc6e6def01 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
78972024-11-28T02:07:56.802ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 50b485b6-2abb-4b68-b3e7-ade51bde3e7a roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
78982024-11-28T02:07:56.803ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 50b485b6-2abb-4b68-b3e7-ade51bde3e7a resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
78992024-11-28T02:07:56.805ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2470 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 50b485b6-2abb-4b68-b3e7-ade51bde3e7a response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
79002024-11-28T02:07:57.823ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 59644528-ae67-441b-8a77-05c1faf83936 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
79012024-11-28T02:07:57.824ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 59644528-ae67-441b-8a77-05c1faf83936 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
79022024-11-28T02:07:57.826ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2822 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 59644528-ae67-441b-8a77-05c1faf83936 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
79032024-11-28T02:07:58.844ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 760900d7-50a4-4f51-a09c-9cde48408008 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
79042024-11-28T02:07:58.845ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 760900d7-50a4-4f51-a09c-9cde48408008 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
79052024-11-28T02:07:58.847ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2451 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 760900d7-50a4-4f51-a09c-9cde48408008 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
79062024-11-28T02:07:59.866ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 861a5f75-77ef-4418-bcbe-013ac0fc9710 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
79072024-11-28T02:07:59.866ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 861a5f75-77ef-4418-bcbe-013ac0fc9710 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
79082024-11-28T02:07:59.868ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2633 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 861a5f75-77ef-4418-bcbe-013ac0fc9710 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
79092024-11-28T02:08:00.478ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 90 reason = Timeout
79102024-11-28T02:08:00.479ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
79112024-11-28T02:08:00.479ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 90 reason = Timeout
79122024-11-28T02:08:00.479ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
79132024-11-28T02:08:00.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
79142024-11-28T02:08:00.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
79152024-11-28T02:08:00.480ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
79162024-11-28T02:08:00.481ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
79172024-11-28T02:08:00.481ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
79182024-11-28T02:08:00.483ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 4.210013ms iteration = 90 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
79192024-11-28T02:08:00.483ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
79202024-11-28T02:08:00.483ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
79212024-11-28T02:08:00.483ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
79222024-11-28T02:08:00.484ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
79232024-11-28T02:08:00.484ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
79242024-11-28T02:08:00.486ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
79252024-11-28T02:08:00.488ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 67ac247f-c934-4c78-857a-fba4fbeb9937, lookup_type: ById(67ac247f-c934-4c78-857a-fba4fbeb9937) } result = Ok(())
79262024-11-28T02:08:00.488ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
79272024-11-28T02:08:00.488ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
79282024-11-28T02:08:00.501ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 67ac247f-c934-4c78-857a-fba4fbeb9937 original_time_created = 2024-11-28 01:53:03.159475 UTC target_id = 67ac247f-c934-4c78-857a-fba4fbeb9937 time_created = 2024-11-28 01:53:03.159475 UTC
79292024-11-28T02:08:00.501ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 22.572412ms iteration = 90 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"67ac247f-c934-4c78-857a-fba4fbeb9937","time_created":"2024-11-28T01:53:03.159475Z"}
79302024-11-28T02:08:00.888ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = b0c06f09-d3f1-46d8-94b4-c1a4d6717ad9 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
79312024-11-28T02:08:00.888ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = b0c06f09-d3f1-46d8-94b4-c1a4d6717ad9 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
79322024-11-28T02:08:00.890ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2125 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = b0c06f09-d3f1-46d8-94b4-c1a4d6717ad9 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
79332024-11-28T02:08:01.910ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 3e13b6b5-ea0d-4a30-a541-489dcdcb0877 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
79342024-11-28T02:08:01.910ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 3e13b6b5-ea0d-4a30-a541-489dcdcb0877 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
79352024-11-28T02:08:01.912ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2589 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 3e13b6b5-ea0d-4a30-a541-489dcdcb0877 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
79362024-11-28T02:08:02.931ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 2b5529e8-c415-497e-a219-dc2cf353ef93 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
79372024-11-28T02:08:02.931ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 2b5529e8-c415-497e-a219-dc2cf353ef93 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
79382024-11-28T02:08:02.933ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2465 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 2b5529e8-c415-497e-a219-dc2cf353ef93 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
79392024-11-28T02:08:03.953ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 875dbc73-cb00-4296-b02c-e1d55b72f317 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
79402024-11-28T02:08:03.953ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 875dbc73-cb00-4296-b02c-e1d55b72f317 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
79412024-11-28T02:08:03.955ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2384 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 875dbc73-cb00-4296-b02c-e1d55b72f317 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
79422024-11-28T02:08:04.196ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1731 local_addr = [fd00:1122:3344:101::b]:55621 method = GET remote_addr = [fd00:1122:3344:101::d]:56417 req_id = 19185b82-520d-4590-ade6-a3b6d08fe988 response_code = 200 uri = /2469a748-bcd2-4e0d-99cc-f79def0452cc
79432024-11-28T02:08:04.974ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 66d9b12b-8b36-4c54-bd32-cdf28af709e0 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
79442024-11-28T02:08:04.974ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 66d9b12b-8b36-4c54-bd32-cdf28af709e0 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
79452024-11-28T02:08:04.976ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2267 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 66d9b12b-8b36-4c54-bd32-cdf28af709e0 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
79462024-11-28T02:08:05.995ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = b6d547b7-4762-403e-9713-4da9f233c4ff roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
79472024-11-28T02:08:05.995ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = b6d547b7-4762-403e-9713-4da9f233c4ff resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
79482024-11-28T02:08:05.997ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2686 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = b6d547b7-4762-403e-9713-4da9f233c4ff response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
79492024-11-28T02:08:07.018ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 0429e09d-d38d-40a4-9fac-e29cd058ce55 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
79502024-11-28T02:08:07.018ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 0429e09d-d38d-40a4-9fac-e29cd058ce55 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
79512024-11-28T02:08:07.020ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2371 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 0429e09d-d38d-40a4-9fac-e29cd058ce55 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
79522024-11-28T02:08:08.041ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = f860d06e-ff68-462e-b423-a741970d88f6 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
79532024-11-28T02:08:08.041ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = f860d06e-ff68-462e-b423-a741970d88f6 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
79542024-11-28T02:08:08.043ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2710 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = f860d06e-ff68-462e-b423-a741970d88f6 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
79552024-11-28T02:08:08.712ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:40402 req_id = dd8f56ed-39f8-47ea-815e-e2f4f51699a0 roles = RoleSet { roles: {} } uri = /nat/ipv4/changeset/10?limit=100
79562024-11-28T02:08:08.713ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:40402 req_id = dd8f56ed-39f8-47ea-815e-e2f4f51699a0 resource = Database result = Ok(()) uri = /nat/ipv4/changeset/10?limit=100
79572024-11-28T02:08:08.716ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 3521 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:40402 req_id = dd8f56ed-39f8-47ea-815e-e2f4f51699a0 response_code = 200 uri = /nat/ipv4/changeset/10?limit=100
79582024-11-28T02:08:09.061ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 542edb23-9144-425d-ad3f-0fb93a6ebf66 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
79592024-11-28T02:08:09.061ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 542edb23-9144-425d-ad3f-0fb93a6ebf66 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
79602024-11-28T02:08:09.063ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1927 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 542edb23-9144-425d-ad3f-0fb93a6ebf66 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
79612024-11-28T02:08:09.186ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:55495 req_id = 28f657c0-46f2-4105-836d-286a545560b4 roles = RoleSet { roles: {} } uri = /metrics/collectors/119eed64-2e24-4d5f-8c42-be6cd8bec8b5/producers?sort_by=id_ascending
79622024-11-28T02:08:09.187ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:55495 req_id = 28f657c0-46f2-4105-836d-286a545560b4 resource = Database result = Ok(()) uri = /metrics/collectors/119eed64-2e24-4d5f-8c42-be6cd8bec8b5/producers?sort_by=id_ascending
79632024-11-28T02:08:09.189ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 3331 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:55495 req_id = 28f657c0-46f2-4105-836d-286a545560b4 response_code = 200 uri = /metrics/collectors/119eed64-2e24-4d5f-8c42-be6cd8bec8b5/producers?sort_by=id_ascending
79642024-11-28T02:08:09.190ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:55495 req_id = edfb6504-876a-4de6-9bbd-a420b2c515ba roles = RoleSet { roles: {} } uri = /metrics/collectors/119eed64-2e24-4d5f-8c42-be6cd8bec8b5/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJjNDM0YjQ0OC01MTk5LTQ2NmUtYWYzMi00ZjYyOWUzMWM3YjEifX0%3D
79652024-11-28T02:08:09.190ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:55495 req_id = edfb6504-876a-4de6-9bbd-a420b2c515ba resource = Database result = Ok(()) uri = /metrics/collectors/119eed64-2e24-4d5f-8c42-be6cd8bec8b5/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJjNDM0YjQ0OC01MTk5LTQ2NmUtYWYzMi00ZjYyOWUzMWM3YjEifX0%3D
79662024-11-28T02:08:09.192ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2402 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:55495 req_id = edfb6504-876a-4de6-9bbd-a420b2c515ba response_code = 200 uri = /metrics/collectors/119eed64-2e24-4d5f-8c42-be6cd8bec8b5/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJjNDM0YjQ0OC01MTk5LTQ2NmUtYWYzMi00ZjYyOWUzMWM3YjEifX0%3D
79672024-11-28T02:08:10.064ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::b]:12221 remote_addr = [fd00:1122:3344:101::1]:38195
79682024-11-28T02:08:10.064ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::1]:38195 req_id = 2695e03a-9d2a-45ca-acbf-4015e0d98e06 roles = RoleSet { roles: {} } uri = /metrics/producers
79692024-11-28T02:08:10.064ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::1]:38195 req_id = 2695e03a-9d2a-45ca-acbf-4015e0d98e06 resource = Database result = Ok(()) uri = /metrics/producers
79702024-11-28T02:08:10.082ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 3ab0b761-9fba-4407-99a0-7aedc3944b4a roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
79712024-11-28T02:08:10.082ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 3ab0b761-9fba-4407-99a0-7aedc3944b4a resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
79722024-11-28T02:08:10.084ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2238 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 3ab0b761-9fba-4407-99a0-7aedc3944b4a response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
79732024-11-28T02:08:10.111ZINFOnexus: assigned collector to new producer collector_id = 119eed64-2e24-4d5f-8c42-be6cd8bec8b5 file = nexus/src/app/oximeter.rs:76 producer_id = 313847b3-d670-4167-9b09-9e7d3dfcb2a4
79742024-11-28T02:08:10.111ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 47163 local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::1]:38195 req_id = 2695e03a-9d2a-45ca-acbf-4015e0d98e06 response_code = 201 uri = /metrics/producers
79752024-11-28T02:08:10.479ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 18 reason = Timeout
79762024-11-28T02:08:10.479ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 31 reason = Timeout
79772024-11-28T02:08:10.479ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 17 reason = Timeout
79782024-11-28T02:08:10.479ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
79792024-11-28T02:08:10.479ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
79802024-11-28T02:08:10.479ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 31 reason = Timeout
79812024-11-28T02:08:10.479ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
79822024-11-28T02:08:10.479ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 17 reason = Timeout
79832024-11-28T02:08:10.479ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
79842024-11-28T02:08:10.479ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 31 reason = Timeout
79852024-11-28T02:08:10.479ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
79862024-11-28T02:08:10.479ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
79872024-11-28T02:08:10.479ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
79882024-11-28T02:08:10.479ZDEBGnexus (BackgroundTasks): activating background_task = service_firewall_rule_propagation iteration = 4 reason = Timeout
79892024-11-28T02:08:10.479ZDEBGnexus (service-firewall-rule-progator): starting background task for service firewall rule propagation background_task = service_firewall_rule_propagation
79902024-11-28T02:08:10.479ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
79912024-11-28T02:08:10.479ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 31 reason = Timeout
79922024-11-28T02:08:10.479ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 16 reason = Timeout
79932024-11-28T02:08:10.479ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2024-11-28 01:58:10.479570782 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
79942024-11-28T02:08:10.479ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
79952024-11-28T02:08:10.479ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
79962024-11-28T02:08:10.479ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 31 reason = Timeout
79972024-11-28T02:08:10.479ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
79982024-11-28T02:08:10.479ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 91 reason = Timeout
79992024-11-28T02:08:10.479ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
80002024-11-28T02:08:10.479ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 31 reason = Timeout
80012024-11-28T02:08:10.479ZDEBGnexus (BackgroundTasks): activating background_task = crdb_node_id_collector iteration = 7 reason = Timeout
80022024-11-28T02:08:10.479ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
80032024-11-28T02:08:10.479ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 31 reason = Timeout
80042024-11-28T02:08:10.479ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
80052024-11-28T02:08:10.479ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
80062024-11-28T02:08:10.479ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 17 reason = Timeout
80072024-11-28T02:08:10.479ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 16 reason = Timeout
80082024-11-28T02:08:10.479ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
80092024-11-28T02:08:10.479ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 16 reason = Timeout
80102024-11-28T02:08:10.479ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:56
80112024-11-28T02:08:10.479ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
80122024-11-28T02:08:10.479ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 31 reason = Timeout
80132024-11-28T02:08:10.479ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
80142024-11-28T02:08:10.479ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 17 reason = Timeout
80152024-11-28T02:08:10.479ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 18 reason = Timeout
80162024-11-28T02:08:10.479ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 32 reason = Timeout
80172024-11-28T02:08:10.479ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
80182024-11-28T02:08:10.479ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 91 reason = Timeout
80192024-11-28T02:08:10.479ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
80202024-11-28T02:08:10.479ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 35 reason = Timeout
80212024-11-28T02:08:10.479ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
80222024-11-28T02:08:10.479ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 31 reason = Timeout
80232024-11-28T02:08:10.479ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
80242024-11-28T02:08:10.479ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 16 reason = Timeout
80252024-11-28T02:08:10.479ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 31 reason = Timeout
80262024-11-28T02:08:10.479ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
80272024-11-28T02:08:10.479ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
80282024-11-28T02:08:10.479ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 17 reason = Timeout
80292024-11-28T02:08:10.479ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:83 reason = blueprint disabled target_id = 67ac247f-c934-4c78-857a-fba4fbeb9937
80302024-11-28T02:08:10.479ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 17 reason = Timeout
80312024-11-28T02:08:10.479ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 15.591\u{b5}s iteration = 17 status = {"enabled":false,"target_id":"67ac247f-c934-4c78-857a-fba4fbeb9937"}
80322024-11-28T02:08:10.479ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
80332024-11-28T02:08:10.479ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
80342024-11-28T02:08:10.479ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 31 reason = Timeout
80352024-11-28T02:08:10.479ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 33 reason = Timeout
80362024-11-28T02:08:10.479ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
80372024-11-28T02:08:10.479ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 16 reason = Timeout
80382024-11-28T02:08:10.479ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 31 reason = Timeout
80392024-11-28T02:08:10.480ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
80402024-11-28T02:08:10.480ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
80412024-11-28T02:08:10.480ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
80422024-11-28T02:08:10.480ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
80432024-11-28T02:08:10.480ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
80442024-11-28T02:08:10.480ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
80452024-11-28T02:08:10.480ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
80462024-11-28T02:08:10.480ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
80472024-11-28T02:08:10.480ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
80482024-11-28T02:08:10.480ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
80492024-11-28T02:08:10.480ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
80502024-11-28T02:08:10.480ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
80512024-11-28T02:08:10.480ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
80522024-11-28T02:08:10.480ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
80532024-11-28T02:08:10.480ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
80542024-11-28T02:08:10.480ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
80552024-11-28T02:08:10.480ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
80562024-11-28T02:08:10.480ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
80572024-11-28T02:08:10.480ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("36d4f615-7461-4364-960c-89ece3aaabaf.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("eb274350-c0b0-4bce-9c8a-502880f2c905.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("eb274350-c0b0-4bce-9c8a-502880f2c905.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::8))) }], valid_until: Instant { tv_sec: 1756, tv_nsec: 949991375 } })
80582024-11-28T02:08:10.480ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
80592024-11-28T02:08:10.480ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
80602024-11-28T02:08:10.480ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
80612024-11-28T02:08:10.480ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
80622024-11-28T02:08:10.480ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
80632024-11-28T02:08:10.480ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
80642024-11-28T02:08:10.480ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
80652024-11-28T02:08:10.480ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
80662024-11-28T02:08:10.480ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
80672024-11-28T02:08:10.480ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
80682024-11-28T02:08:10.480ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
80692024-11-28T02:08:10.480ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
80702024-11-28T02:08:10.480ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("41ea4a99-3038-474e-bf28-2ad338f3243f.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("9402a51b-d30f-4544-93ec-0f5dc7a485e4.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("d23bc5f2-6b92-4992-bdd2-8695e5bf9c0f.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("d23bc5f2-6b92-4992-bdd2-8695e5bf9c0f.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:1::1))) }], valid_until: Instant { tv_sec: 1756, tv_nsec: 950183749 } })
80712024-11-28T02:08:10.480ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
80722024-11-28T02:08:10.480ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
80732024-11-28T02:08:10.480ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
80742024-11-28T02:08:10.480ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
80752024-11-28T02:08:10.480ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } dns_group = external
80762024-11-28T02:08:10.480ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.207561ms iteration = 17 status = {"addresses":["[fd00:1122:3344:101::9]:5353","[fd00:1122:3344:101::8]:5353"]}
80772024-11-28T02:08:10.481ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
80782024-11-28T02:08:10.481ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
80792024-11-28T02:08:10.481ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } dns_group = internal
80802024-11-28T02:08:10.481ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.439705ms iteration = 17 status = {"addresses":["[fd00:1122:3344:3::1]:5353","[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:1::1]:5353"]}
80812024-11-28T02:08:10.481ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
80822024-11-28T02:08:10.483ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-313847b3-d670-4167-9b09-9e7d3dfcb2a4.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-313847b3-d670-4167-9b09-9e7d3dfcb2a4.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1756, tv_nsec: 952616871 } })
80832024-11-28T02:08:10.483ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1103
80842024-11-28T02:08:10.483ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.672734ms iteration = 31 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
80852024-11-28T02:08:10.483ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
80862024-11-28T02:08:10.484ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
80872024-11-28T02:08:10.484ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
80882024-11-28T02:08:10.484ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
80892024-11-28T02:08:10.484ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
80902024-11-28T02:08:10.484ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
80912024-11-28T02:08:10.485ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
80922024-11-28T02:08:10.485ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
80932024-11-28T02:08:10.485ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
80942024-11-28T02:08:10.485ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
80952024-11-28T02:08:10.485ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
80962024-11-28T02:08:10.485ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
80972024-11-28T02:08:10.486ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
80982024-11-28T02:08:10.486ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
80992024-11-28T02:08:10.486ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
81002024-11-28T02:08:10.488ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
81012024-11-28T02:08:10.488ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
81022024-11-28T02:08:10.488ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
81032024-11-28T02:08:10.488ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
81042024-11-28T02:08:10.488ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
81052024-11-28T02:08:10.490ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2024-11-28 01:58:10.479570782 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
81062024-11-28T02:08:10.490ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 10.544243ms iteration = 16 status = {"expiration":"2024-11-28T01:58:10.479570782Z","pruned":[]}
81072024-11-28T02:08:10.490ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
81082024-11-28T02:08:10.491ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
81092024-11-28T02:08:10.492ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
81102024-11-28T02:08:10.492ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
81112024-11-28T02:08:10.492ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
81122024-11-28T02:08:10.492ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
81132024-11-28T02:08:10.492ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
81142024-11-28T02:08:10.492ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
81152024-11-28T02:08:10.493ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
81162024-11-28T02:08:10.493ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
81172024-11-28T02:08:10.493ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
81182024-11-28T02:08:10.494ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
81192024-11-28T02:08:10.494ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
81202024-11-28T02:08:10.494ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
81212024-11-28T02:08:10.494ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
81222024-11-28T02:08:10.495ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
81232024-11-28T02:08:10.496ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
81242024-11-28T02:08:10.497ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
81252024-11-28T02:08:10.497ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
81262024-11-28T02:08:10.497ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
81272024-11-28T02:08:10.499ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
81282024-11-28T02:08:10.499ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
81292024-11-28T02:08:10.499ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
81302024-11-28T02:08:10.500ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-313847b3-d670-4167-9b09-9e7d3dfcb2a4.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-313847b3-d670-4167-9b09-9e7d3dfcb2a4.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1756, tv_nsec: 969678967 } })
81312024-11-28T02:08:10.500ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1103
81322024-11-28T02:08:10.501ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 21.702567ms iteration = 16 status = {"errors":[],"found_port_ok":[]}
81332024-11-28T02:08:10.502ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
81342024-11-28T02:08:10.502ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
81352024-11-28T02:08:10.502ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
81362024-11-28T02:08:10.506ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
81372024-11-28T02:08:10.508ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
81382024-11-28T02:08:10.508ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
81392024-11-28T02:08:10.509ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
81402024-11-28T02:08:10.514ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 31.372845ms iteration = 31 status = {"errors":[],"garbage_collect_requested":[]}
81412024-11-28T02:08:10.515ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
81422024-11-28T02:08:10.518ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
81432024-11-28T02:08:10.518ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
81442024-11-28T02:08:10.519ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
81452024-11-28T02:08:10.519ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
81462024-11-28T02:08:10.519ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
81472024-11-28T02:08:10.519ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
81482024-11-28T02:08:10.519ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
81492024-11-28T02:08:10.520ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
81502024-11-28T02:08:10.520ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
81512024-11-28T02:08:10.521ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
81522024-11-28T02:08:10.521ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
81532024-11-28T02:08:10.522ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
81542024-11-28T02:08:10.523ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
81552024-11-28T02:08:10.523ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
81562024-11-28T02:08:10.523ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
81572024-11-28T02:08:10.523ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
81582024-11-28T02:08:10.524ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
81592024-11-28T02:08:10.525ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
81602024-11-28T02:08:10.525ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
81612024-11-28T02:08:10.525ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
81622024-11-28T02:08:10.525ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
81632024-11-28T02:08:10.525ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
81642024-11-28T02:08:10.526ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
81652024-11-28T02:08:10.526ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
81662024-11-28T02:08:10.528ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
81672024-11-28T02:08:10.528ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
81682024-11-28T02:08:10.528ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
81692024-11-28T02:08:10.529ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-313847b3-d670-4167-9b09-9e7d3dfcb2a4.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-313847b3-d670-4167-9b09-9e7d3dfcb2a4.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1756, tv_nsec: 998809233 } })
81702024-11-28T02:08:10.529ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1103 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc
81712024-11-28T02:08:10.531ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 51.353632ms iteration = 91 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
81722024-11-28T02:08:10.532ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
81732024-11-28T02:08:10.533ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
81742024-11-28T02:08:10.533ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
81752024-11-28T02:08:10.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
81762024-11-28T02:08:10.534ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
81772024-11-28T02:08:10.534ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
81782024-11-28T02:08:10.535ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
81792024-11-28T02:08:10.535ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
81802024-11-28T02:08:10.535ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
81812024-11-28T02:08:10.535ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
81822024-11-28T02:08:10.535ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
81832024-11-28T02:08:10.535ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
81842024-11-28T02:08:10.537ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
81852024-11-28T02:08:10.537ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
81862024-11-28T02:08:10.538ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
81872024-11-28T02:08:10.539ZDEBGnexus (BackgroundTasks): activation complete background_task = crdb_node_id_collector elapsed = 59.766428ms iteration = 7 status = {"nsuccess":5}
81882024-11-28T02:08:10.540ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
81892024-11-28T02:08:10.541ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 67ac247f-c934-4c78-857a-fba4fbeb9937, lookup_type: ById(67ac247f-c934-4c78-857a-fba4fbeb9937) } result = Ok(())
81902024-11-28T02:08:10.541ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
81912024-11-28T02:08:10.541ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
81922024-11-28T02:08:10.542ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
81932024-11-28T02:08:10.542ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
81942024-11-28T02:08:10.542ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
81952024-11-28T02:08:10.543ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
81962024-11-28T02:08:10.544ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
81972024-11-28T02:08:10.546ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
81982024-11-28T02:08:10.546ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
81992024-11-28T02:08:10.548ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
82002024-11-28T02:08:10.553ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
82012024-11-28T02:08:10.553ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
82022024-11-28T02:08:10.553ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
82032024-11-28T02:08:10.553ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
82042024-11-28T02:08:10.554ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:171 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
82052024-11-28T02:08:10.554ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 73.924884ms iteration = 16 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
82062024-11-28T02:08:10.554ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 74.053726ms iteration = 31 status = {"errors":[],"records_set_to_done":[]}
82072024-11-28T02:08:10.556ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
82082024-11-28T02:08:10.556ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
82092024-11-28T02:08:10.558ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 67ac247f-c934-4c78-857a-fba4fbeb9937 original_time_created = 2024-11-28 01:53:03.159475 UTC target_id = 67ac247f-c934-4c78-857a-fba4fbeb9937 time_created = 2024-11-28 01:53:03.159475 UTC
82102024-11-28T02:08:10.558ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 79.179365ms iteration = 91 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"67ac247f-c934-4c78-857a-fba4fbeb9937","time_created":"2024-11-28T01:53:03.159475Z"}
82112024-11-28T02:08:10.562ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
82122024-11-28T02:08:10.562ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
82132024-11-28T02:08:10.566ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 86.698356ms iteration = 31 status = {"errors":[],"requests_created_ok":[],"start_invoked_ok":[]}
82142024-11-28T02:08:10.566ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
82152024-11-28T02:08:10.566ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
82162024-11-28T02:08:10.567ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
82172024-11-28T02:08:10.567ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
82182024-11-28T02:08:10.567ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
82192024-11-28T02:08:10.567ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
82202024-11-28T02:08:10.567ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
82212024-11-28T02:08:10.573ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
82222024-11-28T02:08:10.573ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
82232024-11-28T02:08:10.573ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
82242024-11-28T02:08:10.574ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
82252024-11-28T02:08:10.575ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
82262024-11-28T02:08:10.576ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
82272024-11-28T02:08:10.577ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
82282024-11-28T02:08:10.577ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
82292024-11-28T02:08:10.577ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
82302024-11-28T02:08:10.577ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
82312024-11-28T02:08:10.577ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
82322024-11-28T02:08:10.578ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
82332024-11-28T02:08:10.579ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
82342024-11-28T02:08:10.579ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
82352024-11-28T02:08:10.579ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
82362024-11-28T02:08:10.579ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
82372024-11-28T02:08:10.581ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
82382024-11-28T02:08:10.581ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
82392024-11-28T02:08:10.581ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
82402024-11-28T02:08:10.582ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
82412024-11-28T02:08:10.582ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
82422024-11-28T02:08:10.582ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
82432024-11-28T02:08:10.582ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 102.850726ms iteration = 31 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
82442024-11-28T02:08:10.582ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
82452024-11-28T02:08:10.583ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
82462024-11-28T02:08:10.584ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
82472024-11-28T02:08:10.584ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
82482024-11-28T02:08:10.585ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
82492024-11-28T02:08:10.586ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
82502024-11-28T02:08:10.586ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:140 found = 0 not_ready_to_be_deleted = 0
82512024-11-28T02:08:10.586ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 106.520951ms iteration = 16 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
82522024-11-28T02:08:10.586ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
82532024-11-28T02:08:10.586ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
82542024-11-28T02:08:10.586ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
82552024-11-28T02:08:10.586ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
82562024-11-28T02:08:10.586ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
82572024-11-28T02:08:10.588ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 313847b3-d670-4167-9b09-9e7d3dfcb2a4, lookup_type: ById(313847b3-d670-4167-9b09-9e7d3dfcb2a4) } result = Ok(())
82582024-11-28T02:08:10.588ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
82592024-11-28T02:08:10.588ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
82602024-11-28T02:08:10.588ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
82612024-11-28T02:08:10.589ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
82622024-11-28T02:08:10.590ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
82632024-11-28T02:08:10.590ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
82642024-11-28T02:08:10.590ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
82652024-11-28T02:08:10.590ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 111.10467ms iteration = 31 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
82662024-11-28T02:08:10.591ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 111.942065ms iteration = 31 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[]}
82672024-11-28T02:08:10.592ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 113.143015ms iteration = 31 status = 0
82682024-11-28T02:08:10.593ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
82692024-11-28T02:08:10.593ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
82702024-11-28T02:08:10.594ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
82712024-11-28T02:08:10.594ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
82722024-11-28T02:08:10.596ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
82732024-11-28T02:08:10.596ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
82742024-11-28T02:08:10.597ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
82752024-11-28T02:08:10.602ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
82762024-11-28T02:08:10.602ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
82772024-11-28T02:08:10.602ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
82782024-11-28T02:08:10.604ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
82792024-11-28T02:08:10.604ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
82802024-11-28T02:08:10.604ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
82812024-11-28T02:08:10.604ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
82822024-11-28T02:08:10.606ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
82832024-11-28T02:08:10.607ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
82842024-11-28T02:08:10.609ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
82852024-11-28T02:08:10.609ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
82862024-11-28T02:08:10.611ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
82872024-11-28T02:08:10.612ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
82882024-11-28T02:08:10.614ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
82892024-11-28T02:08:10.614ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
82902024-11-28T02:08:10.619ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
82912024-11-28T02:08:10.619ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
82922024-11-28T02:08:10.619ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 140.326878ms iteration = 31 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
82932024-11-28T02:08:10.620ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
82942024-11-28T02:08:10.620ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 808a324e-c913-4bfc-91a5-d43c961cd511 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
82952024-11-28T02:08:10.620ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
82962024-11-28T02:08:10.620ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
82972024-11-28T02:08:10.622ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
82982024-11-28T02:08:10.624ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
82992024-11-28T02:08:10.624ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
83002024-11-28T02:08:10.624ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
83012024-11-28T02:08:10.624ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
83022024-11-28T02:08:10.624ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
83032024-11-28T02:08:10.624ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = a7e6af61-408c-483d-b043-c27ccf55708e dns_zone_name = oxide.test found_so_far = 0 version = 2
83042024-11-28T02:08:10.624ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
83052024-11-28T02:08:10.624ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
83062024-11-28T02:08:10.625ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
83072024-11-28T02:08:10.625ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
83082024-11-28T02:08:10.626ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
83092024-11-28T02:08:10.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
83102024-11-28T02:08:10.626ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
83112024-11-28T02:08:10.626ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
83122024-11-28T02:08:10.627ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 808a324e-c913-4bfc-91a5-d43c961cd511 dns_zone_name = control-plane.oxide.internal found_so_far = 51 version = 1
83132024-11-28T02:08:10.627ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
83142024-11-28T02:08:10.627ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2024-11-28 01:53:42.502562 UTC dns_group = internal generation = 1
83152024-11-28T02:08:10.627ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 148.096592ms iteration = 17 status = {"generation":1}
83162024-11-28T02:08:10.628ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
83172024-11-28T02:08:10.628ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
83182024-11-28T02:08:10.629ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
83192024-11-28T02:08:10.630ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
83202024-11-28T02:08:10.630ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
83212024-11-28T02:08:10.630ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
83222024-11-28T02:08:10.630ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
83232024-11-28T02:08:10.630ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
83242024-11-28T02:08:10.630ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
83252024-11-28T02:08:10.631ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
83262024-11-28T02:08:10.632ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = a7e6af61-408c-483d-b043-c27ccf55708e dns_zone_name = oxide.test found_so_far = 1 version = 2
83272024-11-28T02:08:10.632ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
83282024-11-28T02:08:10.632ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2024-11-28 01:53:43.351579 UTC dns_group = external generation = 2
83292024-11-28T02:08:10.632ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 152.874505ms iteration = 17 status = {"generation":2}
83302024-11-28T02:08:10.633ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
83312024-11-28T02:08:10.633ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
83322024-11-28T02:08:10.633ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
83332024-11-28T02:08:10.649ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
83342024-11-28T02:08:10.650ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
83352024-11-28T02:08:10.651ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
83362024-11-28T02:08:10.656ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
83372024-11-28T02:08:10.656ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
83382024-11-28T02:08:10.656ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
83392024-11-28T02:08:10.658ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
83402024-11-28T02:08:10.658ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
83412024-11-28T02:08:10.676ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2024-11-28T01:53:08.890632Z, time_modified: 2024-11-28T01:53:08.890632Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 7e159834-997a-4c03-b9db-1e91d783221c, db_silo: Silo { identity: SiloIdentity { id: 7e159834-997a-4c03-b9db-1e91d783221c, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-11-28T01:53:43.308696Z, time_modified: 2024-11-28T01:53:43.308696Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "bc39d1b3b4bb84a49d6458ed7aedc5bca9e77742c7b35882b64d188c0602d306" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 7e159834-997a-4c03-b9db-1e91d783221c, db_silo: Silo { identity: SiloIdentity { id: 7e159834-997a-4c03-b9db-1e91d783221c, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-11-28T01:53:43.308696Z, time_modified: 2024-11-28T01:53:43.308696Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "bc39d1b3b4bb84a49d6458ed7aedc5bca9e77742c7b35882b64d188c0602d306" }] }) }
83422024-11-28T02:08:10.676ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 196.91046ms iteration = 17 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"7e159834-997a-4c03-b9db-1e91d783221c","tls_certs":["bc39d1b3b4bb84a49d6458ed7aedc5bca9e77742c7b35882b64d188c0602d306"]}},"default_endpoint":{"silo_id":"7e159834-997a-4c03-b9db-1e91d783221c","tls_certs":["bc39d1b3b4bb84a49d6458ed7aedc5bca9e77742c7b35882b64d188c0602d306"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
83432024-11-28T02:08:10.677ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 197.43289ms iteration = 35 status = {"physical_disks_added":0}
83442024-11-28T02:08:10.721ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1111 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc zone_address = fd00:1122:3344:101::2
83452024-11-28T02:08:10.721ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
83462024-11-28T02:08:10.721ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "471708d4-261a-403a-b460-996cffbbfe77", "content-length": "26", "date": "Thu, 28 Nov 2024 02:08:10 GMT"} })
83472024-11-28T02:08:10.721ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1116 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
83482024-11-28T02:08:10.723ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc
83492024-11-28T02:08:10.735ZWARNnexus (BackgroundTasks): instance reincarnation completed with errors background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:103 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0 query_errors = 0 restart_errors = 0
83502024-11-28T02:08:10.735ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 255.878505ms iteration = 16 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
83512024-11-28T02:08:10.753ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
83522024-11-28T02:08:10.844ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
83532024-11-28T02:08:10.894ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
83542024-11-28T02:08:10.894ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
83552024-11-28T02:08:10.895ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "67d8e47e-249c-49e2-846e-077ed9e8c466", "content-length": "26", "date": "Thu, 28 Nov 2024 02:08:10 GMT"} })
83562024-11-28T02:08:10.895ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
83572024-11-28T02:08:10.897ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
83582024-11-28T02:08:10.900ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
83592024-11-28T02:08:10.900ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
83602024-11-28T02:08:10.900ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "597b11a3-ed81-4b0a-a1ab-4c8641838117", "content-length": "26", "date": "Thu, 28 Nov 2024 02:08:10 GMT"} })
83612024-11-28T02:08:10.900ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
83622024-11-28T02:08:10.901ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
83632024-11-28T02:08:11.066ZINFOnexus (BackgroundTasks): VPC route manager sled 313847b3-d670-4167-9b09-9e7d3dfcb2a4 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
83642024-11-28T02:08:11.066ZDEBGnexus (BackgroundTasks): client request SledAgent = 313847b3-d670-4167-9b09-9e7d3dfcb2a4 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
83652024-11-28T02:08:11.068ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
83662024-11-28T02:08:11.068ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
83672024-11-28T02:08:11.070ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
83682024-11-28T02:08:11.073ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
83692024-11-28T02:08:11.073ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
83702024-11-28T02:08:11.073ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
83712024-11-28T02:08:11.078ZDEBGnexus (BackgroundTasks): client request SledAgent = 313847b3-d670-4167-9b09-9e7d3dfcb2a4 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
83722024-11-28T02:08:11.083ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
83732024-11-28T02:08:11.083ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
83742024-11-28T02:08:11.085ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
83752024-11-28T02:08:11.085ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
83762024-11-28T02:08:11.087ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
83772024-11-28T02:08:11.088ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
83782024-11-28T02:08:11.090ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
83792024-11-28T02:08:11.090ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
83802024-11-28T02:08:11.092ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
83812024-11-28T02:08:11.092ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
83822024-11-28T02:08:11.094ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
83832024-11-28T02:08:11.094ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
83842024-11-28T02:08:11.096ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
83852024-11-28T02:08:11.096ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
83862024-11-28T02:08:11.097ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
83872024-11-28T02:08:11.101ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
83882024-11-28T02:08:11.101ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
83892024-11-28T02:08:11.101ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
83902024-11-28T02:08:11.102ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
83912024-11-28T02:08:11.102ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
83922024-11-28T02:08:11.104ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
83932024-11-28T02:08:11.104ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
83942024-11-28T02:08:11.105ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
83952024-11-28T02:08:11.109ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
83962024-11-28T02:08:11.109ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
83972024-11-28T02:08:11.109ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
83982024-11-28T02:08:11.112ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
83992024-11-28T02:08:11.112ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "346c1a57-7ff0-4dff-a746-631bac48504b", "content-length": "2", "date": "Thu, 28 Nov 2024 02:08:10 GMT"} })
84002024-11-28T02:08:11.114ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
84012024-11-28T02:08:11.114ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
84022024-11-28T02:08:11.115ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
84032024-11-28T02:08:11.115ZDEBGnexus (service-firewall-rule-progator): resolved names for firewall rules background_task = service_firewall_rule_propagation instance_interfaces = {} subnet_interfaces = {Name("external-dns"): [NetworkInterface { id: bdebcae5-93f8-4dda-99a8-d4a7f51c845c, kind: Service { id: 36d4f615-7461-4364-960c-89ece3aaabaf }, name: Name("external-dns-36d4f615-7461-4364-960c-89ece3aaabaf"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 140, 1])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: cc93e5ec-9515-444a-bf62-fb07fd9a5d9d, kind: Service { id: eb274350-c0b0-4bce-9c8a-502880f2c905 }, name: Name("external-dns-eb274350-c0b0-4bce-9c8a-502880f2c905"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 133, 17])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], Name("nexus"): [NetworkInterface { id: d932a551-47e5-4093-9823-e79bc6fd3354, kind: Service { id: 6392fcec-0540-4c8e-8fb9-a456637e79d1 }, name: Name("nexus-6392fcec-0540-4c8e-8fb9-a456637e79d1"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 167, 213])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: f17d445e-bb27-44c7-a47e-78951fc5205d, kind: Service { id: 2d033909-79ed-49ae-b11d-aeb32740cc17 }, name: Name("nexus-2d033909-79ed-49ae-b11d-aeb32740cc17"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 221, 212])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: fa7a6292-a407-4f82-8188-c5971cd240ed, kind: Service { id: 2469a748-bcd2-4e0d-99cc-f79def0452cc }, name: Name("nexus-2469a748-bcd2-4e0d-99cc-f79def0452cc"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 228, 85])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }]} subnet_networks = {Name("nexus"): [V4(Ipv4Network { addr: 172.30.2.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:2::, prefix: 64 })], Name("external-dns"): [V4(Ipv4Network { addr: 172.30.1.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:1::, prefix: 64 })]} vpc_interfaces = {}
84042024-11-28T02:08:11.115ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
84052024-11-28T02:08:11.116ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 636.216591ms iteration = 31 status = null
84062024-11-28T02:08:11.116ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
84072024-11-28T02:08:11.118ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
84082024-11-28T02:08:11.119ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Fleet result = Ok(())
84092024-11-28T02:08:11.119ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
84102024-11-28T02:08:11.120ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
84112024-11-28T02:08:11.121ZDEBGnexus (service-firewall-rule-progator): Allowlist for user-facing services is set to allow any inbound traffic. Existing VPC firewall rules will not be modified. background_task = service_firewall_rule_propagation
84122024-11-28T02:08:11.121ZDEBGnexus (service-firewall-rule-progator): resolved firewall rules for sled agents background_task = service_firewall_rule_propagation sled_agent_rules = [ResolvedVpcFirewallRule { action: Allow, direction: Inbound, filter_hosts: None, filter_ports: Some([L4PortRange("53")]), filter_protocols: Some([Udp]), priority: 65534, status: Enabled, targets: [NetworkInterface { id: bdebcae5-93f8-4dda-99a8-d4a7f51c845c, kind: Service { id: 36d4f615-7461-4364-960c-89ece3aaabaf }, name: Name("external-dns-36d4f615-7461-4364-960c-89ece3aaabaf"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 140, 1])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: cc93e5ec-9515-444a-bf62-fb07fd9a5d9d, kind: Service { id: eb274350-c0b0-4bce-9c8a-502880f2c905 }, name: Name("external-dns-eb274350-c0b0-4bce-9c8a-502880f2c905"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 133, 17])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }] }, ResolvedVpcFirewallRule { action: Allow, direction: Inbound, filter_hosts: None, filter_ports: Some([L4PortRange("80"), L4PortRange("443")]), filter_protocols: Some([Tcp]), priority: 65534, status: Enabled, targets: [NetworkInterface { id: d932a551-47e5-4093-9823-e79bc6fd3354, kind: Service { id: 6392fcec-0540-4c8e-8fb9-a456637e79d1 }, name: Name("nexus-6392fcec-0540-4c8e-8fb9-a456637e79d1"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 167, 213])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: f17d445e-bb27-44c7-a47e-78951fc5205d, kind: Service { id: 2d033909-79ed-49ae-b11d-aeb32740cc17 }, name: Name("nexus-2d033909-79ed-49ae-b11d-aeb32740cc17"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 221, 212])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: fa7a6292-a407-4f82-8188-c5971cd240ed, kind: Service { id: 2469a748-bcd2-4e0d-99cc-f79def0452cc }, name: Name("nexus-2469a748-bcd2-4e0d-99cc-f79def0452cc"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 228, 85])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }] }]
84132024-11-28T02:08:11.121ZDEBGnexus (service-firewall-rule-progator): resolved 2 rules for sleds background_task = service_firewall_rule_propagation
84142024-11-28T02:08:11.140ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
84152024-11-28T02:08:11.140ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "b9a88fc3-e5fb-419e-b375-ac0dfa93e1f6", "date": "Thu, 28 Nov 2024 02:08:10 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
84162024-11-28T02:08:11.140ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
84172024-11-28T02:08:11.142ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "f45dc4e9-bb46-4bcb-b2da-3e4edce8c124", "date": "Thu, 28 Nov 2024 02:08:10 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
84182024-11-28T02:08:11.142ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
84192024-11-28T02:08:11.144ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
84202024-11-28T02:08:11.144ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 664.973511ms iteration = 18 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
84212024-11-28T02:08:11.164ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = b99a887e-935c-4332-b27f-28c6b376e7f2 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
84222024-11-28T02:08:11.164ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = b99a887e-935c-4332-b27f-28c6b376e7f2 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
84232024-11-28T02:08:11.164ZDEBGnexus (BackgroundTasks): client response SledAgent = 313847b3-d670-4167-9b09-9e7d3dfcb2a4 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "3c6b3ba6-8fce-454e-931f-e8eee00daeb2", "content-length": "372", "date": "Thu, 28 Nov 2024 02:08:10 GMT"} })
84242024-11-28T02:08:11.164ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
84252024-11-28T02:08:11.164ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
84262024-11-28T02:08:11.164ZDEBGnexus (BackgroundTasks): client response SledAgent = 313847b3-d670-4167-9b09-9e7d3dfcb2a4 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "28a06ded-db7c-4ac4-8718-9bcb85d075a5", "content-length": "691", "date": "Thu, 28 Nov 2024 02:08:11 GMT"} })
84272024-11-28T02:08:11.164ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
84282024-11-28T02:08:11.164ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
84292024-11-28T02:08:11.164ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
84302024-11-28T02:08:11.166ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2063 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = b99a887e-935c-4332-b27f-28c6b376e7f2 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
84312024-11-28T02:08:11.166ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 686.942672ms iteration = 32 status = {}
84322024-11-28T02:08:11.179ZDEBGnexus (service-firewall-rule-progator): resolved sleds for vpc oxide-services background_task = service_firewall_rule_propagation vpc_to_sled = [Sled { identity: SledIdentity { id: 313847b3-d670-4167-9b09-9e7d3dfcb2a4, time_created: 2024-11-28T01:53:42.307381Z, time_modified: 2024-11-28T01:53:42.307381Z }, time_deleted: None, rcgen: Generation(Generation(19)), rack_id: 92a8ff6e-21d3-4c28-bcad-e323847985dc, is_scrimlet: true, serial_number: "sock", part_number: "i86pc", revision: SqlU32(0), usable_hardware_threads: SqlU32(48), usable_physical_ram: ByteCount(ByteCount(102952026112)), reservoir_size: ByteCount(ByteCount(51474595840)), ip: fd00:1122:3344:101::1, port: SqlU16(12345), last_used_address: fd00:1122:3344:101::1:0, policy: InService, state: Active, sled_agent_gen: Generation(Generation(1)) }]
84332024-11-28T02:08:11.179ZDEBGnexus (service-firewall-rule-progator): sending firewall rules to sled agents background_task = service_firewall_rule_propagation
84342024-11-28T02:08:11.179ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
84352024-11-28T02:08:11.179ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
84362024-11-28T02:08:11.180ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
84372024-11-28T02:08:11.180ZDEBGnexus (BackgroundTasks): client request SledAgent = 313847b3-d670-4167-9b09-9e7d3dfcb2a4 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
84382024-11-28T02:08:11.181ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
84392024-11-28T02:08:11.181ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
84402024-11-28T02:08:11.183ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
84412024-11-28T02:08:11.184ZDEBGnexus (BackgroundTasks): client response SledAgent = 313847b3-d670-4167-9b09-9e7d3dfcb2a4 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "46b95df9-c19b-427f-b6be-5da408908498", "date": "Thu, 28 Nov 2024 02:08:11 GMT"} })
84422024-11-28T02:08:11.184ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
84432024-11-28T02:08:11.184ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
84442024-11-28T02:08:11.186ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Sled { parent: Fleet, key: 313847b3-d670-4167-9b09-9e7d3dfcb2a4, lookup_type: ById(313847b3-d670-4167-9b09-9e7d3dfcb2a4) } result = Ok(())
84452024-11-28T02:08:11.186ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
84462024-11-28T02:08:11.186ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
84472024-11-28T02:08:11.189ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
84482024-11-28T02:08:11.189ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
84492024-11-28T02:08:11.192ZINFOnexus (BackgroundTasks): VPC route manager sled 313847b3-d670-4167-9b09-9e7d3dfcb2a4 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
84502024-11-28T02:08:11.192ZDEBGnexus (BackgroundTasks): client request SledAgent = 313847b3-d670-4167-9b09-9e7d3dfcb2a4 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
84512024-11-28T02:08:11.195ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 715.866874ms iteration = 31 status = {}
84522024-11-28T02:08:11.197ZDEBGnexus (BackgroundTasks): client response SledAgent = 313847b3-d670-4167-9b09-9e7d3dfcb2a4 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "f14c227a-a44c-47fa-95cd-07b8feeab962", "date": "Thu, 28 Nov 2024 02:08:11 GMT"} })
84532024-11-28T02:08:11.199ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 719.218642ms iteration = 33 status = {}
84542024-11-28T02:08:11.217ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
84552024-11-28T02:08:11.218ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "d4917408-757f-4160-9048-665997fb21f5", "date": "Thu, 28 Nov 2024 02:08:11 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
84562024-11-28T02:08:11.218ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
84572024-11-28T02:08:11.219ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "8f7c0d90-d306-45ae-8094-a24ba9c5c0d2", "date": "Thu, 28 Nov 2024 02:08:11 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
84582024-11-28T02:08:11.219ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
84592024-11-28T02:08:11.220ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "851c2979-067d-4fdd-9238-9a9c3f3feb16", "date": "Thu, 28 Nov 2024 02:08:11 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
84602024-11-28T02:08:11.220ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
84612024-11-28T02:08:11.223ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
84622024-11-28T02:08:11.223ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 743.287271ms iteration = 18 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
84632024-11-28T02:08:11.259ZDEBGnexus (service-firewall-rule-progator): client request SledAgent = 313847b3-d670-4167-9b09-9e7d3dfcb2a4 background_task = service_firewall_rule_propagation body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules
84642024-11-28T02:08:11.260ZDEBGnexus (service-firewall-rule-progator): client response SledAgent = 313847b3-d670-4167-9b09-9e7d3dfcb2a4 background_task = service_firewall_rule_propagation result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules", status: 204, headers: {"x-request-id": "d6e54d86-4328-4753-94e4-9a442be3395e", "date": "Thu, 28 Nov 2024 02:08:11 GMT"} })
84652024-11-28T02:08:11.261ZINFOnexus (service-firewall-rule-progator): updated firewall rules on 1 sleds background_task = service_firewall_rule_propagation file = nexus/networking/src/firewall_rules.rs:496
84662024-11-28T02:08:11.261ZDEBGnexus (service-firewall-rule-progator): successfully propagated service firewall rules background_task = service_firewall_rule_propagation elapsed = 781.721778ms
84672024-11-28T02:08:11.261ZDEBGnexus (BackgroundTasks): activation complete background_task = service_firewall_rule_propagation elapsed = 781.732719ms iteration = 4 status = {}
84682024-11-28T02:08:11.262ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
84692024-11-28T02:08:11.262ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
84702024-11-28T02:08:11.264ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 4bb2986f-2e92-4ac3-9dc1-591b6bb60b3e, rack_id: 92a8ff6e-21d3-4c28-bcad-e323847985dc, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(ea300318-9d62-42ed-80ea-21c86590a166) } rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc switch_location = Switch0
84712024-11-28T02:08:11.264ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
84722024-11-28T02:08:11.264ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
84732024-11-28T02:08:11.271ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
84742024-11-28T02:08:11.271ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "c3d3b945-f146-458a-9cda-683b685bc30f", "content-length": "134", "date": "Thu, 28 Nov 2024 02:08:10 GMT"} })
84752024-11-28T02:08:11.271ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1746 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
84762024-11-28T02:08:11.271ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1773 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = ea300318-9d62-42ed-80ea-21c86590a166
84772024-11-28T02:08:11.271ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
84782024-11-28T02:08:11.271ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
84792024-11-28T02:08:11.271ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
84802024-11-28T02:08:11.271ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
84812024-11-28T02:08:11.271ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
84822024-11-28T02:08:11.271ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1920 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
84832024-11-28T02:08:11.272ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
84842024-11-28T02:08:11.272ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
84852024-11-28T02:08:11.272ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc
84862024-11-28T02:08:11.272ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
84872024-11-28T02:08:11.272ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
84882024-11-28T02:08:11.274ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
84892024-11-28T02:08:11.274ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "c0452b40-5d07-4600-915d-cb656816afee", "content-length": "2", "date": "Thu, 28 Nov 2024 02:08:10 GMT"} })
84902024-11-28T02:08:11.274ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
84912024-11-28T02:08:11.274ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "1c250d0e-5a11-428e-b393-ea2a0286d2e5", "content-length": "52", "date": "Thu, 28 Nov 2024 02:08:10 GMT"} })
84922024-11-28T02:08:11.274ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc switch_location = Switch0
84932024-11-28T02:08:11.274ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
84942024-11-28T02:08:11.360ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "2e926c9a-498d-4061-96d1-58d513e55bec", "date": "Thu, 28 Nov 2024 02:08:11 GMT"} })
84952024-11-28T02:08:11.360ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc switch = Switch0
84962024-11-28T02:08:11.360ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
84972024-11-28T02:08:11.360ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 92a8ff6e-21d3-4c28-bcad-e323847985dc result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "1404d95b-b879-4d86-a341-afcbcd306c68", "content-length": "682", "date": "Thu, 28 Nov 2024 02:08:11 GMT"} })
84982024-11-28T02:08:11.360ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
84992024-11-28T02:08:11.361ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
85002024-11-28T02:08:11.364ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
85012024-11-28T02:08:11.365ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
85022024-11-28T02:08:11.367ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
85032024-11-28T02:08:11.368ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
85042024-11-28T02:08:11.370ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
85052024-11-28T02:08:11.370ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
85062024-11-28T02:08:11.379ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 899.488723ms iteration = 31 status = {}
85072024-11-28T02:08:11.963ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::b]:12221 remote_addr = [fd00:1122:3344:101::2]:62911
85082024-11-28T02:08:11.963ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:62911 req_id = 39b0837a-e5e5-4c68-b8b5-f994c2c25d59 roles = RoleSet { roles: {} } uri = /metrics/producers
85092024-11-28T02:08:11.963ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:62911 req_id = 39b0837a-e5e5-4c68-b8b5-f994c2c25d59 resource = Database result = Ok(()) uri = /metrics/producers
85102024-11-28T02:08:11.976ZINFOnexus: assigned collector to new producer collector_id = 119eed64-2e24-4d5f-8c42-be6cd8bec8b5 file = nexus/src/app/oximeter.rs:76 producer_id = 4ea85f4f-c0ac-4d9e-ae1a-e4f649a8809c
85112024-11-28T02:08:11.976ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 13403 local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:62911 req_id = 39b0837a-e5e5-4c68-b8b5-f994c2c25d59 response_code = 201 uri = /metrics/producers
85122024-11-28T02:08:12.192ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 0198164c-07e2-4fe9-a4d0-0c097f7c7345 roles = RoleSet { roles: {} } uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
85132024-11-28T02:08:12.192ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 0198164c-07e2-4fe9-a4d0-0c097f7c7345 resource = Database result = Ok(()) uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
85142024-11-28T02:08:12.194ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2446 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:53948 req_id = 0198164c-07e2-4fe9-a4d0-0c097f7c7345 response_code = 200 uri = /probes/313847b3-d670-4167-9b09-9e7d3dfcb2a4
85152024-11-28T02:08:12.198ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::b]:12221 remote_addr = [fd00:1122:3344:101::1]:54352
85162024-11-28T02:08:12.198ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::1]:54352 req_id = ccbc3c66-f29c-423f-84b9-96a29f69873d roles = RoleSet { roles: {} } uri = /metrics/producers
85172024-11-28T02:08:12.198ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::1]:54352 req_id = ccbc3c66-f29c-423f-84b9-96a29f69873d resource = Database result = Ok(()) uri = /metrics/producers
85182024-11-28T02:08:12.237ZINFOnexus: assigned collector to new producer collector_id = 119eed64-2e24-4d5f-8c42-be6cd8bec8b5 file = nexus/src/app/oximeter.rs:76 producer_id = 7d8f995c-f158-4cd9-9fae-a9ac17bfe5ed
85192024-11-28T02:08:12.237ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 39407 local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::1]:54352 req_id = ccbc3c66-f29c-423f-84b9-96a29f69873d response_code = 201 uri = /metrics/producers
85202024-11-28T02:08:12.775ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::b]:12221 remote_addr = [fd00:1122:3344:101::2]:36722
85212024-11-28T02:08:12.776ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:36722 req_id = e9920a6f-8f2c-401c-aec7-c117627e2c55 roles = RoleSet { roles: {} } uri = /metrics/producers
85222024-11-28T02:08:12.776ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:36722 req_id = e9920a6f-8f2c-401c-aec7-c117627e2c55 resource = Database result = Ok(()) uri = /metrics/producers
85232024-11-28T02:08:12.816ZINFOnexus: assigned collector to new producer collector_id = 119eed64-2e24-4d5f-8c42-be6cd8bec8b5 file = nexus/src/app/oximeter.rs:76 producer_id = 37e16e5f-8832-402e-b7dd-2f5b87405438
85242024-11-28T02:08:12.816ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 40617 local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:36722 req_id = e9920a6f-8f2c-401c-aec7-c117627e2c55 response_code = 201 uri = /metrics/producers