Max level shown:
12025-03-25T23:00:01.995ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 63 reason = Timeout
22025-03-25T23:00:01.995ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
32025-03-25T23:00:01.995ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
42025-03-25T23:00:01.996ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 63 reason = Timeout
52025-03-25T23:00:01.996ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
62025-03-25T23:00:01.997ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
72025-03-25T23:00:01.997ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
82025-03-25T23:00:02.000ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
92025-03-25T23:00:02.000ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
102025-03-25T23:00:02.000ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
112025-03-25T23:00:02.003ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
122025-03-25T23:00:02.003ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
132025-03-25T23:00:02.005ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
142025-03-25T23:00:02.008ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 5d9d6f43-fd71-4256-b06d-1ed4c87e191f, lookup_type: ById(5d9d6f43-fd71-4256-b06d-1ed4c87e191f) } result = Ok(())
152025-03-25T23:00:02.008ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
162025-03-25T23:00:02.008ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
172025-03-25T23:00:02.019ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 5d9d6f43-fd71-4256-b06d-1ed4c87e191f original_time_created = 2025-03-25 22:49:35.383803 UTC target_id = 5d9d6f43-fd71-4256-b06d-1ed4c87e191f time_created = 2025-03-25 22:49:35.383803 UTC
182025-03-25T23:00:02.019ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 23.682074ms iteration = 63 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"5d9d6f43-fd71-4256-b06d-1ed4c87e191f","time_created":"2025-03-25T22:49:35.383803Z"}
192025-03-25T23:00:02.028ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
202025-03-25T23:00:02.028ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
212025-03-25T23:00:02.060ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 64.419543ms iteration = 63 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
222025-03-25T23:00:02.439ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = ce5bbfab-38de-420a-886d-3dbd7658a0ed roles = RoleSet { roles: {} } uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
232025-03-25T23:00:02.439ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = ce5bbfab-38de-420a-886d-3dbd7658a0ed resource = Database result = Ok(()) uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
242025-03-25T23:00:02.441ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2343 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = ce5bbfab-38de-420a-886d-3dbd7658a0ed response_code = 200 uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
252025-03-25T23:00:03.466ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 9e771fa8-e040-48b7-af20-f50cb1a89eeb roles = RoleSet { roles: {} } uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
262025-03-25T23:00:03.466ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 9e771fa8-e040-48b7-af20-f50cb1a89eeb resource = Database result = Ok(()) uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
272025-03-25T23:00:03.468ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2124 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 9e771fa8-e040-48b7-af20-f50cb1a89eeb response_code = 200 uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
282025-03-25T23:00:03.856ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 581 local_addr = [fd00:1122:3344:101::a]:62174 method = GET remote_addr = [fd00:1122:3344:101::d]:63115 req_id = f5e72495-0ec8-44b4-9d4a-000ed853d80e response_code = 200 uri = /8f96aade-61ab-4710-a1b1-0efefa6704ee
292025-03-25T23:00:04.489ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 02f83d7b-71ca-477c-afd5-a67643190714 roles = RoleSet { roles: {} } uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
302025-03-25T23:00:04.489ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 02f83d7b-71ca-477c-afd5-a67643190714 resource = Database result = Ok(()) uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
312025-03-25T23:00:04.492ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 3047 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 02f83d7b-71ca-477c-afd5-a67643190714 response_code = 200 uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
322025-03-25T23:00:05.523ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 66ea4ac7-6df7-4c90-948e-6381aaedf32c roles = RoleSet { roles: {} } uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
332025-03-25T23:00:05.524ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 66ea4ac7-6df7-4c90-948e-6381aaedf32c resource = Database result = Ok(()) uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
342025-03-25T23:00:05.526ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2979 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 66ea4ac7-6df7-4c90-948e-6381aaedf32c response_code = 200 uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
352025-03-25T23:00:06.556ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 13bc93b7-8c6f-469f-bfad-5158e1371906 roles = RoleSet { roles: {} } uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
362025-03-25T23:00:06.557ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 13bc93b7-8c6f-469f-bfad-5158e1371906 resource = Database result = Ok(()) uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
372025-03-25T23:00:06.559ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2208 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 13bc93b7-8c6f-469f-bfad-5158e1371906 response_code = 200 uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
382025-03-25T23:00:07.580ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = eab715df-da01-4edd-a24d-2a8e54b9af3d roles = RoleSet { roles: {} } uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
392025-03-25T23:00:07.580ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = eab715df-da01-4edd-a24d-2a8e54b9af3d resource = Database result = Ok(()) uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
402025-03-25T23:00:07.583ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2443 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = eab715df-da01-4edd-a24d-2a8e54b9af3d response_code = 200 uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
412025-03-25T23:00:08.605ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 31a0e147-2742-44a8-a20b-e6c5cd212216 roles = RoleSet { roles: {} } uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
422025-03-25T23:00:08.606ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 31a0e147-2742-44a8-a20b-e6c5cd212216 resource = Database result = Ok(()) uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
432025-03-25T23:00:08.608ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2182 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 31a0e147-2742-44a8-a20b-e6c5cd212216 response_code = 200 uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
442025-03-25T23:00:08.840ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:44379 req_id = 61a62d42-6abe-4766-b7bb-6c07356b663c roles = RoleSet { roles: {} } uri = /metrics/collectors/aa92c5e2-2c4d-4351-b468-f1bc420a118d/producers?sort_by=id_ascending
452025-03-25T23:00:08.841ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:44379 req_id = 61a62d42-6abe-4766-b7bb-6c07356b663c resource = Database result = Ok(()) uri = /metrics/collectors/aa92c5e2-2c4d-4351-b468-f1bc420a118d/producers?sort_by=id_ascending
462025-03-25T23:00:08.844ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 3629 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:44379 req_id = 61a62d42-6abe-4766-b7bb-6c07356b663c response_code = 200 uri = /metrics/collectors/aa92c5e2-2c4d-4351-b468-f1bc420a118d/producers?sort_by=id_ascending
472025-03-25T23:00:08.844ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:44379 req_id = c4c843ba-fe05-45c3-bd0c-8e91096deec9 roles = RoleSet { roles: {} } uri = /metrics/collectors/aa92c5e2-2c4d-4351-b468-f1bc420a118d/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMzQ4MTE5Yy05YTdjLTRiNWQtOGNkOC1mZWRkZjQ3NjU5ZDUifX0%3D
482025-03-25T23:00:08.844ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:44379 req_id = c4c843ba-fe05-45c3-bd0c-8e91096deec9 resource = Database result = Ok(()) uri = /metrics/collectors/aa92c5e2-2c4d-4351-b468-f1bc420a118d/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMzQ4MTE5Yy05YTdjLTRiNWQtOGNkOC1mZWRkZjQ3NjU5ZDUifX0%3D
492025-03-25T23:00:08.846ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2537 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:44379 req_id = c4c843ba-fe05-45c3-bd0c-8e91096deec9 response_code = 200 uri = /metrics/collectors/aa92c5e2-2c4d-4351-b468-f1bc420a118d/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMzQ4MTE5Yy05YTdjLTRiNWQtOGNkOC1mZWRkZjQ3NjU5ZDUifX0%3D
502025-03-25T23:00:09.629ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = a047b1c1-9f65-47ba-85b2-50853a9dda41 roles = RoleSet { roles: {} } uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
512025-03-25T23:00:09.629ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = a047b1c1-9f65-47ba-85b2-50853a9dda41 resource = Database result = Ok(()) uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
522025-03-25T23:00:09.630ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1892 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = a047b1c1-9f65-47ba-85b2-50853a9dda41 response_code = 200 uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
532025-03-25T23:00:10.652ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = a04a5c5f-ff15-4eb1-a5ff-104c5864e62a roles = RoleSet { roles: {} } uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
542025-03-25T23:00:10.652ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = a04a5c5f-ff15-4eb1-a5ff-104c5864e62a resource = Database result = Ok(()) uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
552025-03-25T23:00:10.654ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2422 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = a04a5c5f-ff15-4eb1-a5ff-104c5864e62a response_code = 200 uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
562025-03-25T23:00:11.678ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 49e7dbbf-0d50-4351-b350-a1faaecdc31c roles = RoleSet { roles: {} } uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
572025-03-25T23:00:11.678ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 49e7dbbf-0d50-4351-b350-a1faaecdc31c resource = Database result = Ok(()) uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
582025-03-25T23:00:11.680ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2537 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 49e7dbbf-0d50-4351-b350-a1faaecdc31c response_code = 200 uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
592025-03-25T23:00:11.954ZINFOnexus (dropshot_internal): received new VMM runtime state from sled agent actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true file = nexus/src/app/instance.rs:2120 local_addr = [fd00:1122:3344:101::a]:12221 method = PUT migration_state = Migrations { migration_in: None, migration_out: None } propolis_id = 3214a533-86bc-409a-9c53-0114031452e2 remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 38d9a7d0-f908-4c9c-bdfb-b43b733dea92 uri = /vmms/3214a533-86bc-409a-9c53-0114031452e2 vmm_state = VmmRuntimeState { state: Starting, gen: Generation(3), time_updated: 2025-03-25T23:00:11.932557685Z }
602025-03-25T23:00:11.954ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 38d9a7d0-f908-4c9c-bdfb-b43b733dea92 roles = RoleSet { roles: {} } uri = /vmms/3214a533-86bc-409a-9c53-0114031452e2
612025-03-25T23:00:11.955ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 38d9a7d0-f908-4c9c-bdfb-b43b733dea92 resource = Database result = Ok(()) uri = /vmms/3214a533-86bc-409a-9c53-0114031452e2
622025-03-25T23:00:11.996ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 64 reason = Timeout
632025-03-25T23:00:11.996ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 22 reason = Timeout
642025-03-25T23:00:11.996ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
652025-03-25T23:00:11.996ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
662025-03-25T23:00:11.996ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 22 reason = Timeout
672025-03-25T23:00:11.996ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 22 reason = Timeout
682025-03-25T23:00:11.996ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
692025-03-25T23:00:11.996ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
702025-03-25T23:00:11.996ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
712025-03-25T23:00:11.996ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
722025-03-25T23:00:11.997ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 22 reason = Timeout
732025-03-25T23:00:11.997ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 64 reason = Timeout
742025-03-25T23:00:11.997ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
752025-03-25T23:00:11.997ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 22 reason = Timeout
762025-03-25T23:00:11.997ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
772025-03-25T23:00:11.997ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
782025-03-25T23:00:11.997ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
792025-03-25T23:00:11.997ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 22 reason = Timeout
802025-03-25T23:00:11.997ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
812025-03-25T23:00:11.997ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 22 reason = Timeout
822025-03-25T23:00:11.997ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
832025-03-25T23:00:11.997ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
842025-03-25T23:00:11.997ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 22 reason = Timeout
852025-03-25T23:00:11.997ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 22 reason = Timeout
862025-03-25T23:00:11.997ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
872025-03-25T23:00:11.997ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
882025-03-25T23:00:11.997ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 22 reason = Timeout
892025-03-25T23:00:11.997ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
902025-03-25T23:00:11.997ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 22 reason = Timeout
912025-03-25T23:00:11.997ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 22 reason = Timeout
922025-03-25T23:00:11.997ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
932025-03-25T23:00:11.997ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
942025-03-25T23:00:11.997ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 22 reason = Timeout
952025-03-25T23:00:11.997ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 22 reason = Timeout
962025-03-25T23:00:11.997ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
972025-03-25T23:00:11.997ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 22 reason = Timeout
982025-03-25T23:00:11.997ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
992025-03-25T23:00:11.997ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
1002025-03-25T23:00:11.997ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 25 reason = Timeout
1012025-03-25T23:00:11.997ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
1022025-03-25T23:00:11.997ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 22 reason = Timeout
1032025-03-25T23:00:11.997ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
1042025-03-25T23:00:11.997ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
1052025-03-25T23:00:11.997ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
1062025-03-25T23:00:11.997ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
1072025-03-25T23:00:11.997ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
1082025-03-25T23:00:11.997ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
1092025-03-25T23:00:11.997ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
1102025-03-25T23:00:11.997ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
1112025-03-25T23:00:11.997ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
1122025-03-25T23:00:11.997ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
1132025-03-25T23:00:11.997ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
1142025-03-25T23:00:11.997ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
1152025-03-25T23:00:11.997ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
1162025-03-25T23:00:11.997ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
1172025-03-25T23:00:11.997ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
1182025-03-25T23:00:11.998ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
1192025-03-25T23:00:11.998ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1202025-03-25T23:00:11.999ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-9af3e790-030d-4974-9b9e-c35f829f8643.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-9af3e790-030d-4974-9b9e-c35f829f8643.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1785, tv_nsec: 445222686 } })
1212025-03-25T23:00:11.999ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-9af3e790-030d-4974-9b9e-c35f829f8643.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-9af3e790-030d-4974-9b9e-c35f829f8643.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1785, tv_nsec: 445231776 } })
1222025-03-25T23:00:11.999ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1077
1232025-03-25T23:00:12.000ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 4.194535ms iteration = 22 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
1242025-03-25T23:00:12.000ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1077
1252025-03-25T23:00:12.001ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
1262025-03-25T23:00:12.001ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
1272025-03-25T23:00:12.002ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
1282025-03-25T23:00:12.003ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1292025-03-25T23:00:12.006ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
1302025-03-25T23:00:12.006ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
1312025-03-25T23:00:12.006ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
1322025-03-25T23:00:12.008ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
1332025-03-25T23:00:12.008ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
1342025-03-25T23:00:12.010ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1352025-03-25T23:00:12.011ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1362025-03-25T23:00:12.012ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
1372025-03-25T23:00:12.013ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
1382025-03-25T23:00:12.013ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
1392025-03-25T23:00:12.014ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
1402025-03-25T23:00:12.014ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
1412025-03-25T23:00:12.014ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
1422025-03-25T23:00:12.014ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 81745 local_addr = [fd00:1122:3344:101::a]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 38d9a7d0-f908-4c9c-bdfb-b43b733dea92 response_code = 204 uri = /vmms/3214a533-86bc-409a-9c53-0114031452e2
1432025-03-25T23:00:12.015ZINFOnexus (dropshot_internal): received new VMM runtime state from sled agent actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true file = nexus/src/app/instance.rs:2120 local_addr = [fd00:1122:3344:101::a]:12221 method = PUT migration_state = Migrations { migration_in: None, migration_out: None } propolis_id = 3214a533-86bc-409a-9c53-0114031452e2 remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 2aa9b5fb-6703-420a-8c10-9a97197ecb31 uri = /vmms/3214a533-86bc-409a-9c53-0114031452e2 vmm_state = VmmRuntimeState { state: Running, gen: Generation(4), time_updated: 2025-03-25T23:00:12.015112346Z }
1442025-03-25T23:00:12.015ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 2aa9b5fb-6703-420a-8c10-9a97197ecb31 roles = RoleSet { roles: {} } uri = /vmms/3214a533-86bc-409a-9c53-0114031452e2
1452025-03-25T23:00:12.015ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 2aa9b5fb-6703-420a-8c10-9a97197ecb31 resource = Database result = Ok(()) uri = /vmms/3214a533-86bc-409a-9c53-0114031452e2
1462025-03-25T23:00:12.016ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
1472025-03-25T23:00:12.016ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
1482025-03-25T23:00:12.017ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-9af3e790-030d-4974-9b9e-c35f829f8643.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-9af3e790-030d-4974-9b9e-c35f829f8643.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1785, tv_nsec: 463732263 } })
1492025-03-25T23:00:12.017ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1077 rack_id = 3ef3cc6b-3ffd-47a3-99ef-c57b25776619
1502025-03-25T23:00:12.019ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
1512025-03-25T23:00:12.019ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
1522025-03-25T23:00:12.053ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1532025-03-25T23:00:12.054ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
1542025-03-25T23:00:12.055ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
1552025-03-25T23:00:12.055ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 58.129261ms iteration = 22 status = {"errors":[],"garbage_collect_requested":[]}
1562025-03-25T23:00:12.056ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
1572025-03-25T23:00:12.056ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
1582025-03-25T23:00:12.056ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
1592025-03-25T23:00:12.060ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1602025-03-25T23:00:12.060ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 63.162004ms iteration = 22 status = {"errors":[],"finish_invoked_ok":[]}
1612025-03-25T23:00:12.062ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
1622025-03-25T23:00:12.062ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
1632025-03-25T23:00:12.062ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
1642025-03-25T23:00:12.064ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 67.564863ms iteration = 22 status = {"errors":[],"requests_created_ok":[]}
1652025-03-25T23:00:12.068ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
1662025-03-25T23:00:12.068ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1672025-03-25T23:00:12.068ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
1682025-03-25T23:00:12.068ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
1692025-03-25T23:00:12.068ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
1702025-03-25T23:00:12.068ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
1712025-03-25T23:00:12.069ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
1722025-03-25T23:00:12.070ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1732025-03-25T23:00:12.070ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 72.962765ms iteration = 22 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
1742025-03-25T23:00:12.070ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
1752025-03-25T23:00:12.070ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
1762025-03-25T23:00:12.070ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
1772025-03-25T23:00:12.071ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
1782025-03-25T23:00:12.072ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
1792025-03-25T23:00:12.072ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
1802025-03-25T23:00:12.072ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
1812025-03-25T23:00:12.072ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
1822025-03-25T23:00:12.072ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
1832025-03-25T23:00:12.073ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
1842025-03-25T23:00:12.077ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1852025-03-25T23:00:12.077ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1085 zone_address = fd00:1122:3344:101::2
1862025-03-25T23:00:12.077ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
1872025-03-25T23:00:12.078ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "eaed7c65-a2f4-4fb5-86fc-2d5492e31273", "content-length": "26", "date": "Tue, 25 Mar 2025 23:00:12 GMT"} })
1882025-03-25T23:00:12.078ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1088 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
1892025-03-25T23:00:12.078ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1902025-03-25T23:00:12.079ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1085 zone_address = fd00:1122:3344:101::2
1912025-03-25T23:00:12.079ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
1922025-03-25T23:00:12.079ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "bbc13e24-ee7c-4636-a2e3-73266be71633", "content-length": "26", "date": "Tue, 25 Mar 2025 23:00:12 GMT"} })
1932025-03-25T23:00:12.079ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1088 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
1942025-03-25T23:00:12.079ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1122 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
1952025-03-25T23:00:12.081ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1122 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
1962025-03-25T23:00:12.082ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
1972025-03-25T23:00:12.082ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
1982025-03-25T23:00:12.082ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 5d9d6f43-fd71-4256-b06d-1ed4c87e191f, lookup_type: ById(5d9d6f43-fd71-4256-b06d-1ed4c87e191f) } result = Ok(())
1992025-03-25T23:00:12.082ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
2002025-03-25T23:00:12.082ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
2012025-03-25T23:00:12.082ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
2022025-03-25T23:00:12.085ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 70037 local_addr = [fd00:1122:3344:101::a]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 2aa9b5fb-6703-420a-8c10-9a97197ecb31 response_code = 204 uri = /vmms/3214a533-86bc-409a-9c53-0114031452e2
2032025-03-25T23:00:12.086ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 89.706483ms iteration = 64 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
2042025-03-25T23:00:12.099ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 102.167283ms iteration = 22 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
2052025-03-25T23:00:12.105ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1085 rack_id = 3ef3cc6b-3ffd-47a3-99ef-c57b25776619 zone_address = fd00:1122:3344:101::2
2062025-03-25T23:00:12.105ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 3ef3cc6b-3ffd-47a3-99ef-c57b25776619 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
2072025-03-25T23:00:12.106ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 3ef3cc6b-3ffd-47a3-99ef-c57b25776619 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "9e75f559-aaed-44cf-ac04-ee24ef148112", "content-length": "26", "date": "Tue, 25 Mar 2025 23:00:12 GMT"} })
2082025-03-25T23:00:12.106ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1088 rack_id = 3ef3cc6b-3ffd-47a3-99ef-c57b25776619 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
2092025-03-25T23:00:12.108ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1122 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 3ef3cc6b-3ffd-47a3-99ef-c57b25776619
2102025-03-25T23:00:12.169ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
2112025-03-25T23:00:12.170ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
2122025-03-25T23:00:12.173ZINFOnexus (BackgroundTasks): VPC route manager sled 9af3e790-030d-4974-9b9e-c35f829f8643 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
2132025-03-25T23:00:12.173ZDEBGnexus (BackgroundTasks): client request SledAgent = 9af3e790-030d-4974-9b9e-c35f829f8643 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
2142025-03-25T23:00:12.173ZDEBGnexus (BackgroundTasks): client response SledAgent = 9af3e790-030d-4974-9b9e-c35f829f8643 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "fe91e1b7-2365-4361-b2d3-7ba65912ad64", "content-length": "585", "date": "Tue, 25 Mar 2025 23:00:12 GMT"} })
2152025-03-25T23:00:12.173ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
2162025-03-25T23:00:12.174ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
2172025-03-25T23:00:12.176ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
2182025-03-25T23:00:12.177ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "0618fe6c-6d21-4d89-868b-c72a7cd4bdc3", "content-length": "1", "date": "Tue, 25 Mar 2025 23:00:11 GMT"} })
2192025-03-25T23:00:12.179ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
2202025-03-25T23:00:12.179ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
2212025-03-25T23:00:12.182ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 186.124146ms iteration = 22 status = null
2222025-03-25T23:00:12.187ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 190.971155ms iteration = 22 status = {}
2232025-03-25T23:00:12.189ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
2242025-03-25T23:00:12.189ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
2252025-03-25T23:00:12.210ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 5d9d6f43-fd71-4256-b06d-1ed4c87e191f original_time_created = 2025-03-25 22:49:35.383803 UTC target_id = 5d9d6f43-fd71-4256-b06d-1ed4c87e191f time_created = 2025-03-25 22:49:35.383803 UTC
2262025-03-25T23:00:12.210ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 214.559047ms iteration = 64 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"5d9d6f43-fd71-4256-b06d-1ed4c87e191f","time_created":"2025-03-25T22:49:35.383803Z"}
2272025-03-25T23:00:12.249ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
2282025-03-25T23:00:12.250ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
2292025-03-25T23:00:12.250ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
2302025-03-25T23:00:12.250ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
2312025-03-25T23:00:12.250ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
2322025-03-25T23:00:12.251ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
2332025-03-25T23:00:12.256ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
2342025-03-25T23:00:12.256ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
2352025-03-25T23:00:12.257ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2362025-03-25T23:00:12.260ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
2372025-03-25T23:00:12.260ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
2382025-03-25T23:00:12.260ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
2392025-03-25T23:00:12.274ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {5518013d-08ec-4edf-8ef0-a1772322635d: {10.151.1.176: {5443e716-1a40-4091-8e79-6b44910106e0}, 10.151.1.175: {5443e716-1a40-4091-8e79-6b44910106e0}}} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = buskin
2402025-03-25T23:00:12.274ZDEBGnexus (BackgroundTasks): client request SledAgent = 9af3e790-030d-4974-9b9e-c35f829f8643 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
2412025-03-25T23:00:12.274ZDEBGnexus (BackgroundTasks): client response SledAgent = 9af3e790-030d-4974-9b9e-c35f829f8643 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "37cc722b-4fe8-4fb0-8927-c8c0598fd19b", "date": "Tue, 25 Mar 2025 23:00:12 GMT"} })
2422025-03-25T23:00:12.274ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
2432025-03-25T23:00:12.274ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
2442025-03-25T23:00:12.286ZDEBGnexus (BackgroundTasks): client request SledAgent = 9af3e790-030d-4974-9b9e-c35f829f8643 background_task = instance_watcher body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vmms/3214a533-86bc-409a-9c53-0114031452e2/state
2452025-03-25T23:00:12.290ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
2462025-03-25T23:00:12.290ZDEBGnexus (BackgroundTasks): client response SledAgent = 9af3e790-030d-4974-9b9e-c35f829f8643 background_task = instance_watcher result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vmms/3214a533-86bc-409a-9c53-0114031452e2/state", status: 200, headers: {"content-type": "application/json", "x-request-id": "3820c4b3-f526-422e-9d22-0f5d66b7c8eb", "content-length": "130", "date": "Tue, 25 Mar 2025 23:00:12 GMT"} })
2472025-03-25T23:00:12.290ZDEBGnexus (BackgroundTasks): updating instance state background_task = instance_watcher instance_id = 42a2133b-3721-4ce4-8ffe-ac478e6f81cd sled_id = 9af3e790-030d-4974-9b9e-c35f829f8643 state = SledVmmState { vmm_state: VmmRuntimeState { state: Running, gen: Generation(4), time_updated: 2025-03-25T23:00:12.015112346Z }, migration_in: None, migration_out: None } vmm_id = 3214a533-86bc-409a-9c53-0114031452e2
2482025-03-25T23:00:12.290ZINFOnexus (BackgroundTasks): received new VMM runtime state from sled agent background_task = instance_watcher file = nexus/src/app/instance.rs:2120 instance_id = 42a2133b-3721-4ce4-8ffe-ac478e6f81cd migration_state = Migrations { migration_in: None, migration_out: None } propolis_id = 3214a533-86bc-409a-9c53-0114031452e2 sled_id = 9af3e790-030d-4974-9b9e-c35f829f8643 vmm_id = 3214a533-86bc-409a-9c53-0114031452e2 vmm_state = VmmRuntimeState { state: Running, gen: Generation(4), time_updated: 2025-03-25T23:00:12.015112346Z }
2492025-03-25T23:00:12.290ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher instance_id = 42a2133b-3721-4ce4-8ffe-ac478e6f81cd roles = RoleSet { roles: {} } sled_id = 9af3e790-030d-4974-9b9e-c35f829f8643 vmm_id = 3214a533-86bc-409a-9c53-0114031452e2
2502025-03-25T23:00:12.290ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
2512025-03-25T23:00:12.290ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher instance_id = 42a2133b-3721-4ce4-8ffe-ac478e6f81cd resource = Database result = Ok(()) sled_id = 9af3e790-030d-4974-9b9e-c35f829f8643 vmm_id = 3214a533-86bc-409a-9c53-0114031452e2
2522025-03-25T23:00:12.291ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
2532025-03-25T23:00:12.291ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
2542025-03-25T23:00:12.292ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2552025-03-25T23:00:12.295ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 9af3e790-030d-4974-9b9e-c35f829f8643, lookup_type: ById(9af3e790-030d-4974-9b9e-c35f829f8643) } result = Ok(())
2562025-03-25T23:00:12.296ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
2572025-03-25T23:00:12.296ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
2582025-03-25T23:00:12.300ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 1 total_failed = 0 total_incomplete = 0 total_instances = 1 update_sagas_queued = 0
2592025-03-25T23:00:12.301ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
2602025-03-25T23:00:12.301ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 304.084124ms iteration = 22 status = 0
2612025-03-25T23:00:12.301ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
2622025-03-25T23:00:12.301ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 304.423172ms iteration = 22 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{"running":1},"pruned_instances":0,"total_instances":1,"update_sagas_queued":0}
2632025-03-25T23:00:12.306ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
2642025-03-25T23:00:12.306ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
2652025-03-25T23:00:12.308ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
2662025-03-25T23:00:12.309ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
2672025-03-25T23:00:12.312ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
2682025-03-25T23:00:12.313ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
2692025-03-25T23:00:12.317ZINFOnexus (BackgroundTasks): VPC route manager sled 9af3e790-030d-4974-9b9e-c35f829f8643 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
2702025-03-25T23:00:12.318ZINFOnexus (BackgroundTasks): VPC route manager sled 9af3e790-030d-4974-9b9e-c35f829f8643 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
2712025-03-25T23:00:12.318ZDEBGnexus (BackgroundTasks): client request SledAgent = 9af3e790-030d-4974-9b9e-c35f829f8643 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
2722025-03-25T23:00:12.318ZDEBGnexus (BackgroundTasks): client response SledAgent = 9af3e790-030d-4974-9b9e-c35f829f8643 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "99bcf38b-e758-431f-9aab-f6d660536614", "date": "Tue, 25 Mar 2025 23:00:12 GMT"} })
2732025-03-25T23:00:12.320ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 323.249987ms iteration = 22 status = {}
2742025-03-25T23:00:12.334ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
2752025-03-25T23:00:12.334ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
2762025-03-25T23:00:12.335ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 338.221534ms iteration = 22 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
2772025-03-25T23:00:12.338ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
2782025-03-25T23:00:12.338ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
2792025-03-25T23:00:12.338ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 341.029707ms iteration = 22 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
2802025-03-25T23:00:12.351ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:360
2812025-03-25T23:00:12.351ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 354.413629ms iteration = 22 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
2822025-03-25T23:00:12.357ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
2832025-03-25T23:00:12.358ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
2842025-03-25T23:00:12.359ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: 25f1877d-a731-48fe-8967-16ef2fb00f3b, rack_id: 3ef3cc6b-3ffd-47a3-99ef-c57b25776619, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(dcb3f9bf-5ea7-47e3-b699-6c3a8c7459c5) } rack_id = 3ef3cc6b-3ffd-47a3-99ef-c57b25776619 switch_location = Switch0
2852025-03-25T23:00:12.359ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
2862025-03-25T23:00:12.360ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
2872025-03-25T23:00:12.363ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 365.878437ms iteration = 25 status = {"physical_disks_added":0}
2882025-03-25T23:00:12.373ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 3ef3cc6b-3ffd-47a3-99ef-c57b25776619 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
2892025-03-25T23:00:12.373ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 3ef3cc6b-3ffd-47a3-99ef-c57b25776619 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "0a8847d4-468e-48cf-bfc3-4a37453073c6", "content-length": "134", "date": "Tue, 25 Mar 2025 23:00:11 GMT"} })
2902025-03-25T23:00:12.373ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1741 rack_id = 3ef3cc6b-3ffd-47a3-99ef-c57b25776619 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
2912025-03-25T23:00:12.373ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1768 rack_id = 3ef3cc6b-3ffd-47a3-99ef-c57b25776619 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = dcb3f9bf-5ea7-47e3-b699-6c3a8c7459c5
2922025-03-25T23:00:12.373ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:378 rack_id = 3ef3cc6b-3ffd-47a3-99ef-c57b25776619 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
2932025-03-25T23:00:12.373ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:382 rack_id = 3ef3cc6b-3ffd-47a3-99ef-c57b25776619 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
2942025-03-25T23:00:12.373ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:391 rack_id = 3ef3cc6b-3ffd-47a3-99ef-c57b25776619 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
2952025-03-25T23:00:12.373ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:397 rack_id = 3ef3cc6b-3ffd-47a3-99ef-c57b25776619 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
2962025-03-25T23:00:12.373ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = 3ef3cc6b-3ffd-47a3-99ef-c57b25776619 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
2972025-03-25T23:00:12.373ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1915 rack_id = 3ef3cc6b-3ffd-47a3-99ef-c57b25776619 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
2982025-03-25T23:00:12.374ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:408 rack_id = 3ef3cc6b-3ffd-47a3-99ef-c57b25776619 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
2992025-03-25T23:00:12.374ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1951 rack_id = 3ef3cc6b-3ffd-47a3-99ef-c57b25776619 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
3002025-03-25T23:00:12.374ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:417 rack_id = 3ef3cc6b-3ffd-47a3-99ef-c57b25776619
3012025-03-25T23:00:12.374ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
3022025-03-25T23:00:12.374ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
3032025-03-25T23:00:12.377ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 3ef3cc6b-3ffd-47a3-99ef-c57b25776619 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
3042025-03-25T23:00:12.377ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 3ef3cc6b-3ffd-47a3-99ef-c57b25776619 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "63186218-64c6-4fee-9284-63e4252ab6b2", "content-length": "2", "date": "Tue, 25 Mar 2025 23:00:11 GMT"} })
3052025-03-25T23:00:12.377ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 3ef3cc6b-3ffd-47a3-99ef-c57b25776619 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
3062025-03-25T23:00:12.377ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 3ef3cc6b-3ffd-47a3-99ef-c57b25776619 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "2ec9421d-178b-47e8-8a26-eb14bb003b5e", "content-length": "52", "date": "Tue, 25 Mar 2025 23:00:11 GMT"} })
3072025-03-25T23:00:12.377ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.1.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:466 rack_id = 3ef3cc6b-3ffd-47a3-99ef-c57b25776619 switch_location = Switch0
3082025-03-25T23:00:12.377ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 3ef3cc6b-3ffd-47a3-99ef-c57b25776619 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
3092025-03-25T23:00:12.387ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3102025-03-25T23:00:12.390ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
3112025-03-25T23:00:12.390ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
3122025-03-25T23:00:12.390ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
3132025-03-25T23:00:12.434ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3142025-03-25T23:00:12.437ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
3152025-03-25T23:00:12.437ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
3162025-03-25T23:00:12.437ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
3172025-03-25T23:00:12.535ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 3ef3cc6b-3ffd-47a3-99ef-c57b25776619 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "e0f6e31b-54bd-4e3e-914d-57f99ae24fa0", "date": "Tue, 25 Mar 2025 23:00:12 GMT"} })
3182025-03-25T23:00:12.535ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:769 rack_id = 3ef3cc6b-3ffd-47a3-99ef-c57b25776619 switch = Switch0
3192025-03-25T23:00:12.535ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 3ef3cc6b-3ffd-47a3-99ef-c57b25776619 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
3202025-03-25T23:00:12.535ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 3ef3cc6b-3ffd-47a3-99ef-c57b25776619 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "8f18da9a-56b3-4788-9fa4-068393edbfe5", "content-length": "682", "date": "Tue, 25 Mar 2025 23:00:11 GMT"} })
3212025-03-25T23:00:12.535ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
3222025-03-25T23:00:12.536ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
3232025-03-25T23:00:12.540ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
3242025-03-25T23:00:12.541ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
3252025-03-25T23:00:12.545ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
3262025-03-25T23:00:12.545ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
3272025-03-25T23:00:12.548ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
3282025-03-25T23:00:12.548ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
3292025-03-25T23:00:12.558ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 561.299713ms iteration = 22 status = {}
3302025-03-25T23:00:12.566ZDEBGnexus (BackgroundTasks): client request SledAgent = 9af3e790-030d-4974-9b9e-c35f829f8643 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
3312025-03-25T23:00:12.689ZDEBGnexus (BackgroundTasks): client response SledAgent = 9af3e790-030d-4974-9b9e-c35f829f8643 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "50fbaec0-671b-4f28-ac92-6a85c2466bda", "content-length": "811", "date": "Tue, 25 Mar 2025 23:00:12 GMT"} })
3322025-03-25T23:00:12.689ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {VirtualNetworkInterfaceHost { physical_host_ip: fd00:1122:3344:101::1, virtual_ip: 172.30.0.5, virtual_mac: MacAddr(MacAddr6([168, 64, 37, 240, 0, 0])), vni: Vni(14194516) }} sled = buskin
3332025-03-25T23:00:12.689ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = buskin
3342025-03-25T23:00:12.689ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = buskin
3352025-03-25T23:00:12.691ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 694.009185ms iteration = 22 status = {}
3362025-03-25T23:00:12.702ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 7a895817-46c2-420b-96e9-f6bad70ef641 roles = RoleSet { roles: {} } uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
3372025-03-25T23:00:12.702ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 7a895817-46c2-420b-96e9-f6bad70ef641 resource = Database result = Ok(()) uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
3382025-03-25T23:00:12.704ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2467 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 7a895817-46c2-420b-96e9-f6bad70ef641 response_code = 200 uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
3392025-03-25T23:00:13.730ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 3b7b4a20-933e-4625-8358-a10946e19362 roles = RoleSet { roles: {} } uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
3402025-03-25T23:00:13.730ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 3b7b4a20-933e-4625-8358-a10946e19362 resource = Database result = Ok(()) uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
3412025-03-25T23:00:13.732ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2900 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 3b7b4a20-933e-4625-8358-a10946e19362 response_code = 200 uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
3422025-03-25T23:00:13.859ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 3286 local_addr = [fd00:1122:3344:101::a]:62174 method = GET remote_addr = [fd00:1122:3344:101::d]:63115 req_id = 4b94f383-bad3-4144-a779-05ef45b9e54f response_code = 200 uri = /8f96aade-61ab-4710-a1b1-0efefa6704ee
3432025-03-25T23:00:14.754ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 5126a3f3-c4b9-4e05-8520-828c5233550c roles = RoleSet { roles: {} } uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
3442025-03-25T23:00:14.754ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 5126a3f3-c4b9-4e05-8520-828c5233550c resource = Database result = Ok(()) uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
3452025-03-25T23:00:14.756ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2135 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 5126a3f3-c4b9-4e05-8520-828c5233550c response_code = 200 uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
3462025-03-25T23:00:15.776ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = df3bd69c-f57a-4bb2-9233-97fc412b37d9 roles = RoleSet { roles: {} } uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
3472025-03-25T23:00:15.777ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = df3bd69c-f57a-4bb2-9233-97fc412b37d9 resource = Database result = Ok(()) uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
3482025-03-25T23:00:15.779ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2901 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = df3bd69c-f57a-4bb2-9233-97fc412b37d9 response_code = 200 uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
3492025-03-25T23:00:16.801ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 9bd341fc-b408-4572-b1f4-a570b153f165 roles = RoleSet { roles: {} } uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
3502025-03-25T23:00:16.801ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 9bd341fc-b408-4572-b1f4-a570b153f165 resource = Database result = Ok(()) uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
3512025-03-25T23:00:16.804ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 3056 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 9bd341fc-b408-4572-b1f4-a570b153f165 response_code = 200 uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
3522025-03-25T23:00:17.826ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 346b3486-4622-4ea1-9b4f-67518eaf199c roles = RoleSet { roles: {} } uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
3532025-03-25T23:00:17.826ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 346b3486-4622-4ea1-9b4f-67518eaf199c resource = Database result = Ok(()) uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
3542025-03-25T23:00:17.828ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2529 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 346b3486-4622-4ea1-9b4f-67518eaf199c response_code = 200 uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
3552025-03-25T23:00:18.849ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 66379928-f97c-4d98-9461-1b3ab6755eb8 roles = RoleSet { roles: {} } uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
3562025-03-25T23:00:18.849ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 66379928-f97c-4d98-9461-1b3ab6755eb8 resource = Database result = Ok(()) uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
3572025-03-25T23:00:18.851ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2488 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 66379928-f97c-4d98-9461-1b3ab6755eb8 response_code = 200 uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
3582025-03-25T23:00:19.872ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = cc1a0aab-3074-4f2a-bb8c-f30ca3349e89 roles = RoleSet { roles: {} } uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
3592025-03-25T23:00:19.873ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = cc1a0aab-3074-4f2a-bb8c-f30ca3349e89 resource = Database result = Ok(()) uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
3602025-03-25T23:00:19.875ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2389 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = cc1a0aab-3074-4f2a-bb8c-f30ca3349e89 response_code = 200 uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
3612025-03-25T23:00:20.894ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = ae245944-cbef-4a96-b3a7-a5ee966a1bbe roles = RoleSet { roles: {} } uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
3622025-03-25T23:00:20.895ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = ae245944-cbef-4a96-b3a7-a5ee966a1bbe resource = Database result = Ok(()) uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
3632025-03-25T23:00:20.897ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2371 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = ae245944-cbef-4a96-b3a7-a5ee966a1bbe response_code = 200 uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
3642025-03-25T23:00:21.680ZINFOnexus (dropshot_internal): received new VMM runtime state from sled agent actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true file = nexus/src/app/instance.rs:2120 local_addr = [fd00:1122:3344:101::a]:12221 method = PUT migration_state = Migrations { migration_in: None, migration_out: None } propolis_id = 3214a533-86bc-409a-9c53-0114031452e2 remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 5a6e0b21-ea13-4e7f-a5a7-4102beb2e78e uri = /vmms/3214a533-86bc-409a-9c53-0114031452e2 vmm_state = VmmRuntimeState { state: Stopping, gen: Generation(6), time_updated: 2025-03-25T23:00:21.680618184Z }
3652025-03-25T23:00:21.681ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 5a6e0b21-ea13-4e7f-a5a7-4102beb2e78e roles = RoleSet { roles: {} } uri = /vmms/3214a533-86bc-409a-9c53-0114031452e2
3662025-03-25T23:00:21.681ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 5a6e0b21-ea13-4e7f-a5a7-4102beb2e78e resource = Database result = Ok(()) uri = /vmms/3214a533-86bc-409a-9c53-0114031452e2
3672025-03-25T23:00:21.737ZWARNnexus (datastore): Retryable transaction failure file = nexus/db-queries/src/transaction_retry.rs:165 retry_after (ms) = 36 transaction = vmm_and_migration_update_runtime
3682025-03-25T23:00:21.780ZINFOnexus (datastore): transaction completed attempts = 2 file = nexus/db-queries/src/transaction_retry.rs:121 transaction = vmm_and_migration_update_runtime
3692025-03-25T23:00:21.780ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 99698 local_addr = [fd00:1122:3344:101::a]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 5a6e0b21-ea13-4e7f-a5a7-4102beb2e78e response_code = 204 uri = /vmms/3214a533-86bc-409a-9c53-0114031452e2
3702025-03-25T23:00:21.919ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 02c049cb-61b2-43ac-b71c-a349ea049e3b roles = RoleSet { roles: {} } uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
3712025-03-25T23:00:21.919ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 02c049cb-61b2-43ac-b71c-a349ea049e3b resource = Database result = Ok(()) uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
3722025-03-25T23:00:21.921ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2111 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 02c049cb-61b2-43ac-b71c-a349ea049e3b response_code = 200 uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
3732025-03-25T23:00:21.995ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 65 reason = Timeout
3742025-03-25T23:00:21.995ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
3752025-03-25T23:00:21.995ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
3762025-03-25T23:00:21.996ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 65 reason = Timeout
3772025-03-25T23:00:21.996ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
3782025-03-25T23:00:21.996ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
3792025-03-25T23:00:21.997ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3802025-03-25T23:00:22.000ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
3812025-03-25T23:00:22.000ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
3822025-03-25T23:00:22.000ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
3832025-03-25T23:00:22.002ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
3842025-03-25T23:00:22.002ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
3852025-03-25T23:00:22.004ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3862025-03-25T23:00:22.007ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 5d9d6f43-fd71-4256-b06d-1ed4c87e191f, lookup_type: ById(5d9d6f43-fd71-4256-b06d-1ed4c87e191f) } result = Ok(())
3872025-03-25T23:00:22.007ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
3882025-03-25T23:00:22.007ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
3892025-03-25T23:00:22.017ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 5d9d6f43-fd71-4256-b06d-1ed4c87e191f original_time_created = 2025-03-25 22:49:35.383803 UTC target_id = 5d9d6f43-fd71-4256-b06d-1ed4c87e191f time_created = 2025-03-25 22:49:35.383803 UTC
3902025-03-25T23:00:22.017ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 22.308672ms iteration = 65 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"5d9d6f43-fd71-4256-b06d-1ed4c87e191f","time_created":"2025-03-25T22:49:35.383803Z"}
3912025-03-25T23:00:22.029ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
3922025-03-25T23:00:22.029ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
3932025-03-25T23:00:22.060ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 63.936881ms iteration = 65 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
3942025-03-25T23:00:22.943ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = e260ef13-13f0-49c5-ac54-ec4adf36a79d roles = RoleSet { roles: {} } uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
3952025-03-25T23:00:22.944ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = e260ef13-13f0-49c5-ac54-ec4adf36a79d resource = Database result = Ok(()) uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
3962025-03-25T23:00:22.945ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2097 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = e260ef13-13f0-49c5-ac54-ec4adf36a79d response_code = 200 uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
3972025-03-25T23:00:23.856ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 591 local_addr = [fd00:1122:3344:101::a]:62174 method = GET remote_addr = [fd00:1122:3344:101::d]:63115 req_id = f5869555-d7fe-4d79-b3c2-05db31f19fd0 response_code = 200 uri = /8f96aade-61ab-4710-a1b1-0efefa6704ee
3982025-03-25T23:00:23.967ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 7d372843-11f6-4b5c-8cb0-0e82a5ada003 roles = RoleSet { roles: {} } uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
3992025-03-25T23:00:23.967ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 7d372843-11f6-4b5c-8cb0-0e82a5ada003 resource = Database result = Ok(()) uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
4002025-03-25T23:00:23.969ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2440 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 7d372843-11f6-4b5c-8cb0-0e82a5ada003 response_code = 200 uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
4012025-03-25T23:00:24.248ZINFOnexus (dropshot_internal): received new VMM runtime state from sled agent actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true file = nexus/src/app/instance.rs:2120 local_addr = [fd00:1122:3344:101::a]:12221 method = PUT migration_state = Migrations { migration_in: None, migration_out: None } propolis_id = 3214a533-86bc-409a-9c53-0114031452e2 remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 46bc7946-5e26-4f8e-84e0-1bca504b91c9 uri = /vmms/3214a533-86bc-409a-9c53-0114031452e2 vmm_state = VmmRuntimeState { state: Destroyed, gen: Generation(8), time_updated: 2025-03-25T23:00:24.247819933Z }
4022025-03-25T23:00:24.248ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 46bc7946-5e26-4f8e-84e0-1bca504b91c9 roles = RoleSet { roles: {} } uri = /vmms/3214a533-86bc-409a-9c53-0114031452e2
4032025-03-25T23:00:24.248ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 46bc7946-5e26-4f8e-84e0-1bca504b91c9 resource = Database result = Ok(()) uri = /vmms/3214a533-86bc-409a-9c53-0114031452e2
4042025-03-25T23:00:24.257ZDEBGnexus (dropshot_internal): new VMM runtime state from sled agent requires an instance-update saga actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = PUT migration_in_needs_update = false migration_out_needs_update = false propolis_id = 3214a533-86bc-409a-9c53-0114031452e2 remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 46bc7946-5e26-4f8e-84e0-1bca504b91c9 uri = /vmms/3214a533-86bc-409a-9c53-0114031452e2 vmm_needs_update = true
4052025-03-25T23:00:24.257ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 46bc7946-5e26-4f8e-84e0-1bca504b91c9 roles = RoleSet { roles: {} } uri = /vmms/3214a533-86bc-409a-9c53-0114031452e2
4062025-03-25T23:00:24.257ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 46bc7946-5e26-4f8e-84e0-1bca504b91c9 resource = Database result = Ok(()) uri = /vmms/3214a533-86bc-409a-9c53-0114031452e2
4072025-03-25T23:00:24.260ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 46bc7946-5e26-4f8e-84e0-1bca504b91c9 roles = RoleSet { roles: {} } uri = /vmms/3214a533-86bc-409a-9c53-0114031452e2
4082025-03-25T23:00:24.260ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 46bc7946-5e26-4f8e-84e0-1bca504b91c9 resource = Database result = Ok(()) uri = /vmms/3214a533-86bc-409a-9c53-0114031452e2
4092025-03-25T23:00:24.262ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 46bc7946-5e26-4f8e-84e0-1bca504b91c9 roles = RoleSet { roles: {} } uri = /vmms/3214a533-86bc-409a-9c53-0114031452e2
4102025-03-25T23:00:24.262ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 46bc7946-5e26-4f8e-84e0-1bca504b91c9 resource = Database result = Ok(()) uri = /vmms/3214a533-86bc-409a-9c53-0114031452e2
4112025-03-25T23:00:24.264ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 46bc7946-5e26-4f8e-84e0-1bca504b91c9 roles = RoleSet { roles: {} } uri = /vmms/3214a533-86bc-409a-9c53-0114031452e2
4122025-03-25T23:00:24.264ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 46bc7946-5e26-4f8e-84e0-1bca504b91c9 resource = Database result = Ok(()) uri = /vmms/3214a533-86bc-409a-9c53-0114031452e2
4132025-03-25T23:00:24.294ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 46bc7946-5e26-4f8e-84e0-1bca504b91c9 roles = RoleSet { roles: {} } uri = /vmms/3214a533-86bc-409a-9c53-0114031452e2
4142025-03-25T23:00:24.294ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 46bc7946-5e26-4f8e-84e0-1bca504b91c9 resource = Database result = Ok(()) uri = /vmms/3214a533-86bc-409a-9c53-0114031452e2
4152025-03-25T23:00:24.295ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 46bc7946-5e26-4f8e-84e0-1bca504b91c9 roles = RoleSet { roles: {} } uri = /vmms/3214a533-86bc-409a-9c53-0114031452e2
4162025-03-25T23:00:24.295ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 46bc7946-5e26-4f8e-84e0-1bca504b91c9 resource = Database result = Ok(()) uri = /vmms/3214a533-86bc-409a-9c53-0114031452e2
4172025-03-25T23:00:24.297ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 46bc7946-5e26-4f8e-84e0-1bca504b91c9 roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } uri = /vmms/3214a533-86bc-409a-9c53-0114031452e2
4182025-03-25T23:00:24.299ZDEBGnexus (dropshot_internal): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 46bc7946-5e26-4f8e-84e0-1bca504b91c9 resource = Instance { parent: Project { parent: Silo { parent: Fleet, key: 526efdec-95ea-41d0-8852-ffb890a1c1a5, lookup_type: ById(526efdec-95ea-41d0-8852-ffb890a1c1a5) }, key: 2fdb0ee2-76b2-4482-8062-671d3c8196a5, lookup_type: ById(2fdb0ee2-76b2-4482-8062-671d3c8196a5) }, key: 42a2133b-3721-4ce4-8ffe-ac478e6f81cd, lookup_type: ById(42a2133b-3721-4ce4-8ffe-ac478e6f81cd) } result = Ok(()) uri = /vmms/3214a533-86bc-409a-9c53-0114031452e2
4192025-03-25T23:00:24.302ZINFOnexus (dropshot_internal): starting update saga for 42a2133b-3721-4ce4-8ffe-ac478e6f81cd actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true file = nexus/src/app/instance.rs:1583 instance_id = 42a2133b-3721-4ce4-8ffe-ac478e6f81cd local_addr = [fd00:1122:3344:101::a]:12221 method = PUT migration_state = Migrations { migration_in: None, migration_out: None } remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 46bc7946-5e26-4f8e-84e0-1bca504b91c9 uri = /vmms/3214a533-86bc-409a-9c53-0114031452e2 vmm_state = VmmRuntimeState { state: Destroyed, gen: Generation(8), time_updated: 2025-03-25T23:00:24.247819933Z }
4202025-03-25T23:00:24.302ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 54762 local_addr = [fd00:1122:3344:101::a]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 46bc7946-5e26-4f8e-84e0-1bca504b91c9 response_code = 204 uri = /vmms/3214a533-86bc-409a-9c53-0114031452e2
4212025-03-25T23:00:24.302ZDEBGnexus (dropshot_internal): preparing instance-update saga for 42a2133b-3721-4ce4-8ffe-ac478e6f81cd... actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true instance_id = 42a2133b-3721-4ce4-8ffe-ac478e6f81cd local_addr = [fd00:1122:3344:101::a]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 46bc7946-5e26-4f8e-84e0-1bca504b91c9 uri = /vmms/3214a533-86bc-409a-9c53-0114031452e2
4222025-03-25T23:00:24.302ZINFOnexus (SagaExecutor): preparing saga file = nexus/src/app/saga.rs:255 saga_id = 1d5c0462-def7-4d55-8913-c2e4e45419a6 saga_name = start-instance-update
4232025-03-25T23:00:24.303ZINFOnexus (dropshot_internal): received new VMM runtime state from sled agent actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true file = nexus/src/app/instance.rs:2120 local_addr = [fd00:1122:3344:101::a]:12221 method = PUT migration_state = Migrations { migration_in: None, migration_out: None } propolis_id = 3214a533-86bc-409a-9c53-0114031452e2 remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 4e00cf64-0b0e-48b0-99e1-65733b08c80c uri = /vmms/3214a533-86bc-409a-9c53-0114031452e2 vmm_state = VmmRuntimeState { state: Destroyed, gen: Generation(8), time_updated: 2025-03-25T23:00:24.247819933Z }
4242025-03-25T23:00:24.303ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 4e00cf64-0b0e-48b0-99e1-65733b08c80c roles = RoleSet { roles: {} } uri = /vmms/3214a533-86bc-409a-9c53-0114031452e2
4252025-03-25T23:00:24.303ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 4e00cf64-0b0e-48b0-99e1-65733b08c80c resource = Database result = Ok(()) uri = /vmms/3214a533-86bc-409a-9c53-0114031452e2
4262025-03-25T23:00:24.308ZINFOnexus (SEC): saga create dag = {"end_node":4,"graph":{"edge_property":"directed","edges":[[0,1,null],[1,2,null],[3,0,null],[2,4,null]],"node_holes":[],"nodes":[{"Action":{"action_name":"common.uuid_generate","label":"GenerateInstanceLockId","name":"saga_instance_lock_id"}},{"Action":{"action_name":"start_instance_update.lock_instance","label":"LockInstance","name":"updater_lock"}},{"Action":{"action_name":"start_instance_update.fetch_state_and_start_real_saga","label":"FetchStateAndStartRealSaga","name":"state"}},{"Start":{"params":{"authz_instance":{"key":"42a2133b-3721-4ce4-8ffe-ac478e6f81cd","lookup_type":{"ById":"42a2133b-3721-4ce4-8ffe-ac478e6f81cd"},"parent":{"key":"2fdb0ee2-76b2-4482-8062-671d3c8196a5","lookup_type":{"ById":"2fdb0ee2-76b2-4482-8062-671d3c8196a5"},"parent":{"key":"526efdec-95ea-41d0-8852-ffb890a1c1a5","lookup_type":{"ById":"526efdec-95ea-41d0-8852-ffb890a1c1a5"},"parent":null}}},"serialized_authn":{"kind":{"Authenticated":[{"actor":{"UserBuiltin":{"user_builtin_id":"001de000-05e4-4000-8000-000000000002"}}},null]}}}}},"End"]},"saga_name":"start-instance-update","start_node":3} file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/steno-0.4.1/src/sec.rs:1146 saga_id = 1d5c0462-def7-4d55-8913-c2e4e45419a6 saga_name = start-instance-update sec_id = 8f96aade-61ab-4710-a1b1-0efefa6704ee
4272025-03-25T23:00:24.308ZINFOnexus (SecStore): creating saga file = nexus/db-queries/src/db/sec_store.rs:50 saga_id = 1d5c0462-def7-4d55-8913-c2e4e45419a6 saga_name = start-instance-update
4282025-03-25T23:00:24.338ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 35284 local_addr = [fd00:1122:3344:101::a]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 4e00cf64-0b0e-48b0-99e1-65733b08c80c response_code = 204 uri = /vmms/3214a533-86bc-409a-9c53-0114031452e2
4292025-03-25T23:00:24.351ZINFOnexus (SagaExecutor): starting saga file = nexus/src/app/saga.rs:353 saga_id = 1d5c0462-def7-4d55-8913-c2e4e45419a6 saga_name = start-instance-update
4302025-03-25T23:00:24.351ZINFOnexus (SEC): saga start file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/steno-0.4.1/src/sec.rs:967 saga_id = 1d5c0462-def7-4d55-8913-c2e4e45419a6 saga_name = start-instance-update sec_id = 8f96aade-61ab-4710-a1b1-0efefa6704ee
4312025-03-25T23:00:24.351ZDEBGnexus (SEC): saga log event new_state = N003 started sec_id = 8f96aade-61ab-4710-a1b1-0efefa6704ee
4322025-03-25T23:00:24.351ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 3 saga_id = 1d5c0462-def7-4d55-8913-c2e4e45419a6
4332025-03-25T23:00:24.356ZDEBGnexus (SEC): saga log event new_state = N003 succeeded sec_id = 8f96aade-61ab-4710-a1b1-0efefa6704ee
4342025-03-25T23:00:24.356ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 3 saga_id = 1d5c0462-def7-4d55-8913-c2e4e45419a6
4352025-03-25T23:00:24.360ZDEBGnexus (SEC): saga log event new_state = N000 started sec_id = 8f96aade-61ab-4710-a1b1-0efefa6704ee
4362025-03-25T23:00:24.360ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 0 saga_id = 1d5c0462-def7-4d55-8913-c2e4e45419a6
4372025-03-25T23:00:24.367ZDEBGnexus (SEC): saga log event new_state = N000 succeeded sec_id = 8f96aade-61ab-4710-a1b1-0efefa6704ee
4382025-03-25T23:00:24.367ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(String("865e2d38-4548-454c-b8f3-af8630bf45fc")) node_id = 0 saga_id = 1d5c0462-def7-4d55-8913-c2e4e45419a6
4392025-03-25T23:00:24.371ZDEBGnexus (SEC): saga log event new_state = N001 started sec_id = 8f96aade-61ab-4710-a1b1-0efefa6704ee
4402025-03-25T23:00:24.371ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 1 saga_id = 1d5c0462-def7-4d55-8913-c2e4e45419a6
4412025-03-25T23:00:24.381ZINFOnexus (SagaExecutor): instance update: attempting to lock instance file = nexus/src/app/sagas/instance_update/start.rs:93 instance_id = 42a2133b-3721-4ce4-8ffe-ac478e6f81cd saga_id = 1d5c0462-def7-4d55-8913-c2e4e45419a6 saga_name = start-instance-update
4422025-03-25T23:00:24.381ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 1d5c0462-def7-4d55-8913-c2e4e45419a6 saga_name = start-instance-update saga_node = LockInstance
4432025-03-25T23:00:24.382ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 1d5c0462-def7-4d55-8913-c2e4e45419a6 saga_name = start-instance-update saga_node = LockInstance
4442025-03-25T23:00:24.383ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 1d5c0462-def7-4d55-8913-c2e4e45419a6 saga_name = start-instance-update saga_node = LockInstance
4452025-03-25T23:00:24.384ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 1d5c0462-def7-4d55-8913-c2e4e45419a6 saga_name = start-instance-update saga_node = LockInstance
4462025-03-25T23:00:24.385ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 1d5c0462-def7-4d55-8913-c2e4e45419a6 saga_name = start-instance-update saga_node = LockInstance
4472025-03-25T23:00:24.386ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 1d5c0462-def7-4d55-8913-c2e4e45419a6 saga_name = start-instance-update saga_node = LockInstance
4482025-03-25T23:00:24.390ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 1d5c0462-def7-4d55-8913-c2e4e45419a6 saga_name = start-instance-update saga_node = LockInstance
4492025-03-25T23:00:24.390ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 1d5c0462-def7-4d55-8913-c2e4e45419a6 saga_name = start-instance-update saga_node = LockInstance
4502025-03-25T23:00:24.391ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 1d5c0462-def7-4d55-8913-c2e4e45419a6 saga_name = start-instance-update saga_node = LockInstance
4512025-03-25T23:00:24.391ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 1d5c0462-def7-4d55-8913-c2e4e45419a6 saga_name = start-instance-update saga_node = LockInstance
4522025-03-25T23:00:24.392ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 1d5c0462-def7-4d55-8913-c2e4e45419a6 saga_name = start-instance-update saga_node = LockInstance
4532025-03-25T23:00:24.392ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 1d5c0462-def7-4d55-8913-c2e4e45419a6 saga_name = start-instance-update saga_node = LockInstance
4542025-03-25T23:00:24.394ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } saga_id = 1d5c0462-def7-4d55-8913-c2e4e45419a6 saga_name = start-instance-update saga_node = LockInstance
4552025-03-25T23:00:24.397ZDEBGnexus (SagaExecutor): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Instance { parent: Project { parent: Silo { parent: Fleet, key: 526efdec-95ea-41d0-8852-ffb890a1c1a5, lookup_type: ById(526efdec-95ea-41d0-8852-ffb890a1c1a5) }, key: 2fdb0ee2-76b2-4482-8062-671d3c8196a5, lookup_type: ById(2fdb0ee2-76b2-4482-8062-671d3c8196a5) }, key: 42a2133b-3721-4ce4-8ffe-ac478e6f81cd, lookup_type: ById(42a2133b-3721-4ce4-8ffe-ac478e6f81cd) } result = Ok(()) saga_id = 1d5c0462-def7-4d55-8913-c2e4e45419a6 saga_name = start-instance-update saga_node = LockInstance
4562025-03-25T23:00:24.397ZDEBGnexus (SagaExecutor): attempting to acquire instance updater lock actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true current_gen = Generation(Generation(1)) instance_id = 42a2133b-3721-4ce4-8ffe-ac478e6f81cd saga_id = 1d5c0462-def7-4d55-8913-c2e4e45419a6 saga_name = start-instance-update saga_node = LockInstance updater_id = 865e2d38-4548-454c-b8f3-af8630bf45fc
4572025-03-25T23:00:24.397ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 1d5c0462-def7-4d55-8913-c2e4e45419a6 saga_name = start-instance-update saga_node = LockInstance
4582025-03-25T23:00:24.397ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 1d5c0462-def7-4d55-8913-c2e4e45419a6 saga_name = start-instance-update saga_node = LockInstance
4592025-03-25T23:00:24.406ZDEBGnexus (SagaExecutor): attempting to acquire instance updater lock actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true current_gen = Generation(Generation(1)) instance_id = 42a2133b-3721-4ce4-8ffe-ac478e6f81cd saga_id = 1d5c0462-def7-4d55-8913-c2e4e45419a6 saga_name = start-instance-update saga_node = LockInstance updater_id = 865e2d38-4548-454c-b8f3-af8630bf45fc
4602025-03-25T23:00:24.406ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 1d5c0462-def7-4d55-8913-c2e4e45419a6 saga_name = start-instance-update saga_node = LockInstance
4612025-03-25T23:00:24.407ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 1d5c0462-def7-4d55-8913-c2e4e45419a6 saga_name = start-instance-update saga_node = LockInstance
4622025-03-25T23:00:24.414ZDEBGnexus (SagaExecutor): instance updater lock acquired! actor_id = 001de000-05e4-4000-8000-000000000002 already_locked = true authenticated = true instance_id = 42a2133b-3721-4ce4-8ffe-ac478e6f81cd locked_gen = Generation(Generation(2)) saga_id = 1d5c0462-def7-4d55-8913-c2e4e45419a6 saga_name = start-instance-update saga_node = LockInstance updater_id = 865e2d38-4548-454c-b8f3-af8630bf45fc
4632025-03-25T23:00:24.414ZDEBGnexus (SEC): saga log event new_state = N001 succeeded sec_id = 8f96aade-61ab-4710-a1b1-0efefa6704ee
4642025-03-25T23:00:24.414ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Object {"locked_gen": Number(2), "updater_id": String("865e2d38-4548-454c-b8f3-af8630bf45fc")}) node_id = 1 saga_id = 1d5c0462-def7-4d55-8913-c2e4e45419a6
4652025-03-25T23:00:24.417ZDEBGnexus (SEC): saga log event new_state = N002 started sec_id = 8f96aade-61ab-4710-a1b1-0efefa6704ee
4662025-03-25T23:00:24.417ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 2 saga_id = 1d5c0462-def7-4d55-8913-c2e4e45419a6
4672025-03-25T23:00:24.425ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 1d5c0462-def7-4d55-8913-c2e4e45419a6 saga_name = start-instance-update saga_node = FetchStateAndStartRealSaga
4682025-03-25T23:00:24.425ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 1d5c0462-def7-4d55-8913-c2e4e45419a6 saga_name = start-instance-update saga_node = FetchStateAndStartRealSaga
4692025-03-25T23:00:24.427ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 1d5c0462-def7-4d55-8913-c2e4e45419a6 saga_name = start-instance-update saga_node = FetchStateAndStartRealSaga
4702025-03-25T23:00:24.427ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 1d5c0462-def7-4d55-8913-c2e4e45419a6 saga_name = start-instance-update saga_node = FetchStateAndStartRealSaga
4712025-03-25T23:00:24.428ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 1d5c0462-def7-4d55-8913-c2e4e45419a6 saga_name = start-instance-update saga_node = FetchStateAndStartRealSaga
4722025-03-25T23:00:24.429ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 1d5c0462-def7-4d55-8913-c2e4e45419a6 saga_name = start-instance-update saga_node = FetchStateAndStartRealSaga
4732025-03-25T23:00:24.430ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } saga_id = 1d5c0462-def7-4d55-8913-c2e4e45419a6 saga_name = start-instance-update saga_node = FetchStateAndStartRealSaga
4742025-03-25T23:00:24.433ZDEBGnexus (SagaExecutor): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Instance { parent: Project { parent: Silo { parent: Fleet, key: 526efdec-95ea-41d0-8852-ffb890a1c1a5, lookup_type: ById(526efdec-95ea-41d0-8852-ffb890a1c1a5) }, key: 2fdb0ee2-76b2-4482-8062-671d3c8196a5, lookup_type: ById(2fdb0ee2-76b2-4482-8062-671d3c8196a5) }, key: 42a2133b-3721-4ce4-8ffe-ac478e6f81cd, lookup_type: ById(42a2133b-3721-4ce4-8ffe-ac478e6f81cd) } result = Ok(()) saga_id = 1d5c0462-def7-4d55-8913-c2e4e45419a6 saga_name = start-instance-update saga_node = FetchStateAndStartRealSaga
4752025-03-25T23:00:24.433ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 1d5c0462-def7-4d55-8913-c2e4e45419a6 saga_name = start-instance-update saga_node = FetchStateAndStartRealSaga
4762025-03-25T23:00:24.433ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 1d5c0462-def7-4d55-8913-c2e4e45419a6 saga_name = start-instance-update saga_node = FetchStateAndStartRealSaga
4772025-03-25T23:00:24.441ZINFOnexus (SagaExecutor): instance update: starting real update saga... current.active_vmm = Some(Vmm { id: 3214a533-86bc-409a-9c53-0114031452e2, time_created: 2025-03-25T23:00:03.380131Z, time_deleted: None, instance_id: 42a2133b-3721-4ce4-8ffe-ac478e6f81cd, sled_id: 9af3e790-030d-4974-9b9e-c35f829f8643, propolis_ip: V6(Ipv6Network { addr: fd00:1122:3344:101::1:0, prefix: 128 }), propolis_port: SqlU16(12400), runtime: VmmRuntimeState { time_state_updated: 2025-03-25T23:00:24.247819Z, gen: Generation(Generation(8)), state: Destroyed } }) current.migration = None current.runtime_state = InstanceRuntimeState { time_updated: 2025-03-25T23:00:00.991038Z, gen: Generation(Generation(3)), propolis_id: Some(3214a533-86bc-409a-9c53-0114031452e2), dst_propolis_id: None, migration_id: None, nexus_state: Vmm, time_last_auto_restarted: None } current.target_vmm = None file = nexus/src/app/sagas/instance_update/start.rs:180 instance_id = 42a2133b-3721-4ce4-8ffe-ac478e6f81cd saga_id = 1d5c0462-def7-4d55-8913-c2e4e45419a6 saga_name = start-instance-update update.deprovision = true update.destroy_active_vmm = Some(3214a533-86bc-409a-9c53-0114031452e2 (propolis)) update.destroy_target_vmm = None update.network_config_update = Some(Delete) update.new_runtime_state = InstanceRuntimeState { time_updated: 2025-03-25T23:00:24.439074494Z, gen: Generation(Generation(4)), propolis_id: None, dst_propolis_id: None, migration_id: None, nexus_state: NoVmm, time_last_auto_restarted: None }
4782025-03-25T23:00:24.446ZINFOnexus (SagaExecutor): preparing saga file = nexus/src/app/saga.rs:255 saga_id = 3d2078bd-1ba9-4423-ac3c-532f2e2601d2 saga_name = instance-update
4792025-03-25T23:00:24.446ZINFOnexus (SEC): saga create dag = {"end_node":14,"graph":{"edge_property":"directed","edges":[[0,1,null],[1,2,null],[2,3,null],[3,4,null],[4,5,null],[5,6,null],[6,7,null],[7,8,null],[9,10,null],[8,9,null],[10,11,null],[11,12,null],[13,0,null],[12,14,null]],"node_holes":[],"nodes":[{"Action":{"action_name":"common.uuid_generate","label":"GenerateInstanceLockId","name":"saga_instance_lock_id"}},{"Action":{"action_name":"instance_update.become_updater","label":"BecomeUpdater","name":"updater_lock"}},{"Constant":{"name":"network_config_update","value":"Delete"}},{"Action":{"action_name":"instance_update.update_network_config","label":"UpdateNetworkConfig","name":"update_network_config"}},{"Action":{"action_name":"instance_update.release_virtual_provisioning","label":"ReleaseVirtualProvisioning","name":"release_virtual_provisioning"}},{"Action":{"action_name":"instance_update.unassign_oximeter_producer","label":"UnassignOximeterProducer","name":"unassign_oximeter_producer"}},{"Action":{"action_name":"instance_update.commit_instance_updates","label":"CommitInstanceUpdates","name":"commit_instance_updates"}},{"Constant":{"name":"destroy_active_vmm_params","value":{"instance_id":"42a2133b-3721-4ce4-8ffe-ac478e6f81cd","serialized_authn":{"kind":{"Authenticated":[{"actor":{"UserBuiltin":{"user_builtin_id":"001de000-05e4-4000-8000-000000000002"}}},null]}},"vmm_id":"3214a533-86bc-409a-9c53-0114031452e2"}}},{"SubsagaStart":{"params_node_name":"destroy_active_vmm_params","saga_name":"destroy_active_vmm"}},{"Action":{"action_name":"destroy_vmm.release_sled_resources","label":"ReleaseSledResources","name":"release_sled_resources"}},{"Action":{"action_name":"destroy_vmm.mark_vmm_deleted","label":"MarkVmmDeleted","name":"mark_vmm_deleted"}},{"SubsagaEnd":{"name":"active_vmm_destroyed"}},{"Action":{"action_name":"instance_update.chain_successor_saga","label":"ChainSuccessorSaga","name":"chain_successor_saga"}},{"Start":{"params":{"authz_instance":{"key":"42a2133b-3721-4ce4-8ffe-ac478e6f81cd","lookup_type":{"ById":"42a2133b-3721-4ce4-8ffe-ac478e6f81cd"},"parent":{"key":"2fdb0ee2-76b2-4482-8062-671d3c8196a5","lookup_type":{"ById":"2fdb0ee2-76b2-4482-8062-671d3c8196a5"},"parent":{"key":"526efdec-95ea-41d0-8852-ffb890a1c1a5","lookup_type":{"ById":"526efdec-95ea-41d0-8852-ffb890a1c1a5"},"parent":null}}},"orig_lock":{"locked_gen":2,"updater_id":"865e2d38-4548-454c-b8f3-af8630bf45fc"},"serialized_authn":{"kind":{"Authenticated":[{"actor":{"UserBuiltin":{"user_builtin_id":"001de000-05e4-4000-8000-000000000002"}}},null]}},"update":{"deprovision":{"cpus_diff":2,"project_id":"2fdb0ee2-76b2-4482-8062-671d3c8196a5","ram_diff":1073741824},"destroy_active_vmm":"3214a533-86bc-409a-9c53-0114031452e2","destroy_target_vmm":null,"network_config":"Delete","new_runtime":{"dst_propolis_id":null,"gen":4,"migration_id":null,"nexus_state":"NoVmm","propolis_id":null,"time_last_auto_restarted":null,"time_updated":"2025-03-25T23:00:24.439074494Z"}}}}},"End"]},"saga_name":"instance-update","start_node":13} file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/steno-0.4.1/src/sec.rs:1146 saga_id = 3d2078bd-1ba9-4423-ac3c-532f2e2601d2 saga_name = instance-update sec_id = 8f96aade-61ab-4710-a1b1-0efefa6704ee
4802025-03-25T23:00:24.446ZINFOnexus (SecStore): creating saga file = nexus/db-queries/src/db/sec_store.rs:50 saga_id = 3d2078bd-1ba9-4423-ac3c-532f2e2601d2 saga_name = instance-update
4812025-03-25T23:00:24.449ZINFOnexus (SagaExecutor): starting saga file = nexus/src/app/saga.rs:353 saga_id = 3d2078bd-1ba9-4423-ac3c-532f2e2601d2 saga_name = instance-update
4822025-03-25T23:00:24.449ZINFOnexus (SEC): saga start file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/steno-0.4.1/src/sec.rs:967 saga_id = 3d2078bd-1ba9-4423-ac3c-532f2e2601d2 saga_name = instance-update sec_id = 8f96aade-61ab-4710-a1b1-0efefa6704ee
4832025-03-25T23:00:24.449ZDEBGnexus (SEC): saga log event new_state = N013 started sec_id = 8f96aade-61ab-4710-a1b1-0efefa6704ee
4842025-03-25T23:00:24.449ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 13 saga_id = 3d2078bd-1ba9-4423-ac3c-532f2e2601d2
4852025-03-25T23:00:24.452ZDEBGnexus (SEC): saga log event new_state = N013 succeeded sec_id = 8f96aade-61ab-4710-a1b1-0efefa6704ee
4862025-03-25T23:00:24.452ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 13 saga_id = 3d2078bd-1ba9-4423-ac3c-532f2e2601d2
4872025-03-25T23:00:24.456ZDEBGnexus (SEC): saga log event new_state = N000 started sec_id = 8f96aade-61ab-4710-a1b1-0efefa6704ee
4882025-03-25T23:00:24.456ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 0 saga_id = 3d2078bd-1ba9-4423-ac3c-532f2e2601d2
4892025-03-25T23:00:24.458ZDEBGnexus (SEC): saga log event new_state = N000 succeeded sec_id = 8f96aade-61ab-4710-a1b1-0efefa6704ee
4902025-03-25T23:00:24.458ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(String("c073b03b-5408-437a-aecc-5a42e2f74e4e")) node_id = 0 saga_id = 3d2078bd-1ba9-4423-ac3c-532f2e2601d2
4912025-03-25T23:00:24.462ZDEBGnexus (SEC): saga log event new_state = N001 started sec_id = 8f96aade-61ab-4710-a1b1-0efefa6704ee
4922025-03-25T23:00:24.462ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 1 saga_id = 3d2078bd-1ba9-4423-ac3c-532f2e2601d2
4932025-03-25T23:00:24.465ZDEBGnexus (SagaExecutor): instance update: trying to become instance updater... instance_id = 42a2133b-3721-4ce4-8ffe-ac478e6f81cd parent_lock = UpdaterLock { updater_id: 865e2d38-4548-454c-b8f3-af8630bf45fc, locked_gen: Generation(Generation(2)) } saga_id = 3d2078bd-1ba9-4423-ac3c-532f2e2601d2 saga_name = instance-update
4942025-03-25T23:00:24.465ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 3d2078bd-1ba9-4423-ac3c-532f2e2601d2 saga_name = instance-update saga_node = BecomeUpdater
4952025-03-25T23:00:24.465ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 3d2078bd-1ba9-4423-ac3c-532f2e2601d2 saga_name = instance-update saga_node = BecomeUpdater
4962025-03-25T23:00:24.473ZDEBGnexus (SagaExecutor): inherited lock from 865e2d38-4548-454c-b8f3-af8630bf45fc to c073b03b-5408-437a-aecc-5a42e2f74e4e actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true instance_id = 42a2133b-3721-4ce4-8ffe-ac478e6f81cd locked_gen = Generation(Generation(3)) parent_gen = Generation(Generation(2)) parent_id = 865e2d38-4548-454c-b8f3-af8630bf45fc saga_id = 3d2078bd-1ba9-4423-ac3c-532f2e2601d2 saga_name = instance-update saga_node = BecomeUpdater updater_id = c073b03b-5408-437a-aecc-5a42e2f74e4e
4972025-03-25T23:00:24.473ZINFOnexus (SagaExecutor): instance_update: Now, I am become Updater, the destroyer of VMMs. file = nexus/src/app/sagas/instance_update/mod.rs:964 instance_id = 42a2133b-3721-4ce4-8ffe-ac478e6f81cd saga_id = 3d2078bd-1ba9-4423-ac3c-532f2e2601d2 saga_name = instance-update
4982025-03-25T23:00:24.473ZDEBGnexus (SEC): saga log event new_state = N001 succeeded sec_id = 8f96aade-61ab-4710-a1b1-0efefa6704ee
4992025-03-25T23:00:24.473ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Object {"locked_gen": Number(3), "updater_id": String("c073b03b-5408-437a-aecc-5a42e2f74e4e")}) node_id = 1 saga_id = 3d2078bd-1ba9-4423-ac3c-532f2e2601d2
5002025-03-25T23:00:24.476ZDEBGnexus (SEC): saga log event new_state = N002 started sec_id = 8f96aade-61ab-4710-a1b1-0efefa6704ee
5012025-03-25T23:00:24.476ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 2 saga_id = 3d2078bd-1ba9-4423-ac3c-532f2e2601d2
5022025-03-25T23:00:24.479ZDEBGnexus (SEC): saga log event new_state = N002 succeeded sec_id = 8f96aade-61ab-4710-a1b1-0efefa6704ee
5032025-03-25T23:00:24.479ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(String("Delete")) node_id = 2 saga_id = 3d2078bd-1ba9-4423-ac3c-532f2e2601d2
5042025-03-25T23:00:24.482ZDEBGnexus (SEC): saga log event new_state = N003 started sec_id = 8f96aade-61ab-4710-a1b1-0efefa6704ee
5052025-03-25T23:00:24.482ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 3 saga_id = 3d2078bd-1ba9-4423-ac3c-532f2e2601d2
5062025-03-25T23:00:24.490ZINFOnexus (SagaExecutor): instance update: deleting network config file = nexus/src/app/sagas/instance_update/mod.rs:1017 instance_id = 42a2133b-3721-4ce4-8ffe-ac478e6f81cd saga_id = 3d2078bd-1ba9-4423-ac3c-532f2e2601d2 saga_name = instance-update
5072025-03-25T23:00:24.490ZINFOnexus: deleting instance dpd configuration file = nexus/src/app/instance_network.rs:548 instance_id = 42a2133b-3721-4ce4-8ffe-ac478e6f81cd
5082025-03-25T23:00:24.490ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 3d2078bd-1ba9-4423-ac3c-532f2e2601d2 saga_name = instance-update saga_node = UpdateNetworkConfig
5092025-03-25T23:00:24.490ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 3d2078bd-1ba9-4423-ac3c-532f2e2601d2 saga_name = instance-update saga_node = UpdateNetworkConfig
5102025-03-25T23:00:24.493ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 3d2078bd-1ba9-4423-ac3c-532f2e2601d2 saga_name = instance-update saga_node = UpdateNetworkConfig
5112025-03-25T23:00:24.494ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 3d2078bd-1ba9-4423-ac3c-532f2e2601d2 saga_name = instance-update saga_node = UpdateNetworkConfig
5122025-03-25T23:00:24.498ZWARNnexus: no matching nat entries to soft delete file = nexus/src/app/instance_network.rs:711
5132025-03-25T23:00:24.498ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 3d2078bd-1ba9-4423-ac3c-532f2e2601d2 saga_name = instance-update saga_node = UpdateNetworkConfig
5142025-03-25T23:00:24.498ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 3d2078bd-1ba9-4423-ac3c-532f2e2601d2 saga_name = instance-update saga_node = UpdateNetworkConfig
5152025-03-25T23:00:24.501ZWARNnexus: no matching nat entries to soft delete file = nexus/src/app/instance_network.rs:711
5162025-03-25T23:00:24.501ZDEBGnexus (InstanceAllocator): roles roles = RoleSet { roles: {} }
5172025-03-25T23:00:24.501ZDEBGnexus (InstanceAllocator): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Database result = Ok(())
5182025-03-25T23:00:24.502ZDEBGnexus (InstanceAllocator): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "viewer")} }
5192025-03-25T23:00:24.503ZDEBGnexus (InstanceAllocator): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Fleet result = Ok(())
5202025-03-25T23:00:24.503ZDEBGnexus (InstanceAllocator): roles roles = RoleSet { roles: {} }
5212025-03-25T23:00:24.503ZDEBGnexus (InstanceAllocator): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Database result = Ok(())
5222025-03-25T23:00:24.508ZDEBGnexus: notifying dendrite of updates instance_id = Some(42a2133b-3721-4ce4-8ffe-ac478e6f81cd (instance)) switch = switch0
5232025-03-25T23:00:24.508ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-9af3e790-030d-4974-9b9e-c35f829f8643.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-9af3e790-030d-4974-9b9e-c35f829f8643.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1797, tv_nsec: 954662503 } })
5242025-03-25T23:00:24.508ZINFOnexus: Determining switch slots managed by switch zones file = nexus/src/app/mod.rs:1077
5252025-03-25T23:00:24.539ZINFOnexus: determining switch slot managed by dendrite zone file = nexus/src/app/mod.rs:1085 zone_address = fd00:1122:3344:101::2
5262025-03-25T23:00:24.539ZDEBGnexus (MgsClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
5272025-03-25T23:00:24.539ZDEBGnexus (MgsClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "5bf28372-6cf0-4e35-a554-3a8b3c40f707", "content-length": "26", "date": "Tue, 25 Mar 2025 23:00:24 GMT"} })
5282025-03-25T23:00:24.539ZINFOnexus: identified switch slot for dendrite zone file = nexus/src/app/mod.rs:1088 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
5292025-03-25T23:00:24.540ZINFOnexus: completed mapping dendrite zones to switch slots file = nexus/src/app/mod.rs:1122 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
5302025-03-25T23:00:24.570ZDEBGnexus (DpdClient): client request body = None method = POST uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/trigger
5312025-03-25T23:00:24.570ZDEBGnexus (DpdClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/trigger", status: 200, headers: {"content-type": "application/json", "x-request-id": "f95a5d5c-9675-4eb3-9060-ac03165769e4", "content-length": "4", "date": "Tue, 25 Mar 2025 23:00:23 GMT"} })
5322025-03-25T23:00:24.571ZDEBGnexus (SEC): saga log event new_state = N003 succeeded sec_id = 8f96aade-61ab-4710-a1b1-0efefa6704ee
5332025-03-25T23:00:24.571ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 3 saga_id = 3d2078bd-1ba9-4423-ac3c-532f2e2601d2
5342025-03-25T23:00:24.575ZDEBGnexus (SEC): saga log event new_state = N004 started sec_id = 8f96aade-61ab-4710-a1b1-0efefa6704ee
5352025-03-25T23:00:24.575ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 4 saga_id = 3d2078bd-1ba9-4423-ac3c-532f2e2601d2
5362025-03-25T23:00:24.580ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 3d2078bd-1ba9-4423-ac3c-532f2e2601d2 saga_name = instance-update saga_node = ReleaseVirtualProvisioning
5372025-03-25T23:00:24.581ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 3d2078bd-1ba9-4423-ac3c-532f2e2601d2 saga_name = instance-update saga_node = ReleaseVirtualProvisioning
5382025-03-25T23:00:24.594ZINFOnexus (SagaExecutor): instance update (no VMM): deallocated virtual provisioning resources file = nexus/src/app/sagas/instance_update/mod.rs:1092 instance_id = 42a2133b-3721-4ce4-8ffe-ac478e6f81cd records_deleted = [VirtualProvisioningCollection { id: 001de000-1334-4000-8000-000000000000, time_modified: Some(2025-03-25T23:00:24.586410Z), collection_type: "Fleet", virtual_disk_bytes_provisioned: ByteCount(ByteCount(6442450944)), cpus_provisioned: 0, ram_provisioned: ByteCount(ByteCount(0)) }, VirtualProvisioningCollection { id: 2fdb0ee2-76b2-4482-8062-671d3c8196a5, time_modified: Some(2025-03-25T23:00:24.586410Z), collection_type: "Project", virtual_disk_bytes_provisioned: ByteCount(ByteCount(2147483648)), cpus_provisioned: 0, ram_provisioned: ByteCount(ByteCount(0)) }, VirtualProvisioningCollection { id: 526efdec-95ea-41d0-8852-ffb890a1c1a5, time_modified: Some(2025-03-25T23:00:24.586410Z), collection_type: "Silo", virtual_disk_bytes_provisioned: ByteCount(ByteCount(6442450944)), cpus_provisioned: 0, ram_provisioned: ByteCount(ByteCount(0)) }] saga_id = 3d2078bd-1ba9-4423-ac3c-532f2e2601d2 saga_name = instance-update
5392025-03-25T23:00:24.594ZDEBGnexus (SEC): saga log event new_state = N004 succeeded sec_id = 8f96aade-61ab-4710-a1b1-0efefa6704ee
5402025-03-25T23:00:24.594ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 4 saga_id = 3d2078bd-1ba9-4423-ac3c-532f2e2601d2
5412025-03-25T23:00:24.598ZDEBGnexus (SEC): saga log event new_state = N005 started sec_id = 8f96aade-61ab-4710-a1b1-0efefa6704ee
5422025-03-25T23:00:24.598ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 5 saga_id = 3d2078bd-1ba9-4423-ac3c-532f2e2601d2
5432025-03-25T23:00:24.601ZINFOnexus (SagaExecutor): instance update (no VMM): unassigning oximeter producer file = nexus/src/app/sagas/instance_update/mod.rs:1127 instance_id = 42a2133b-3721-4ce4-8ffe-ac478e6f81cd saga_id = 3d2078bd-1ba9-4423-ac3c-532f2e2601d2 saga_name = instance-update
5442025-03-25T23:00:24.601ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 3d2078bd-1ba9-4423-ac3c-532f2e2601d2 saga_name = instance-update saga_node = UnassignOximeterProducer
5452025-03-25T23:00:24.601ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 3d2078bd-1ba9-4423-ac3c-532f2e2601d2 saga_name = instance-update saga_node = UnassignOximeterProducer
5462025-03-25T23:00:24.608ZDEBGnexus (SagaExecutor): deleted metric producer assignment collector_id = aa92c5e2-2c4d-4351-b468-f1bc420a118d producer_id = 42a2133b-3721-4ce4-8ffe-ac478e6f81cd saga_id = 3d2078bd-1ba9-4423-ac3c-532f2e2601d2 saga_name = instance-update
5472025-03-25T23:00:24.608ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 3d2078bd-1ba9-4423-ac3c-532f2e2601d2 saga_name = instance-update saga_node = UnassignOximeterProducer
5482025-03-25T23:00:24.608ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 3d2078bd-1ba9-4423-ac3c-532f2e2601d2 saga_name = instance-update saga_node = UnassignOximeterProducer
5492025-03-25T23:00:24.648ZINFOnexus (SagaExecutor): registered oximeter collector client file = nexus/src/app/oximeter.rs:262 id = 42a2133b-3721-4ce4-8ffe-ac478e6f81cd saga_id = 3d2078bd-1ba9-4423-ac3c-532f2e2601d2 saga_name = instance-update
5502025-03-25T23:00:24.648ZDEBGnexus (SagaExecutor): client request body = None method = DELETE oximeter-collector = 42a2133b-3721-4ce4-8ffe-ac478e6f81cd saga_id = 3d2078bd-1ba9-4423-ac3c-532f2e2601d2 saga_name = instance-update uri = http://[fd00:1122:3344:101::d]:12223/producers/42a2133b-3721-4ce4-8ffe-ac478e6f81cd
5512025-03-25T23:00:24.670ZDEBGnexus (SagaExecutor): client response oximeter-collector = 42a2133b-3721-4ce4-8ffe-ac478e6f81cd result = Ok(Response { url: "http://[fd00:1122:3344:101::d]:12223/producers/42a2133b-3721-4ce4-8ffe-ac478e6f81cd", status: 204, headers: {"x-request-id": "16754e99-7118-4e49-8793-8a6de5c6c707", "date": "Tue, 25 Mar 2025 23:00:24 GMT"} }) saga_id = 3d2078bd-1ba9-4423-ac3c-532f2e2601d2 saga_name = instance-update
5522025-03-25T23:00:24.670ZDEBGnexus (SagaExecutor): successfully deleted producer from collector address = [fd00:1122:3344:101::d]:12223 collector_id = aa92c5e2-2c4d-4351-b468-f1bc420a118d producer_id = 42a2133b-3721-4ce4-8ffe-ac478e6f81cd saga_id = 3d2078bd-1ba9-4423-ac3c-532f2e2601d2 saga_name = instance-update
5532025-03-25T23:00:24.671ZDEBGnexus (SEC): saga log event new_state = N005 succeeded sec_id = 8f96aade-61ab-4710-a1b1-0efefa6704ee
5542025-03-25T23:00:24.671ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 5 saga_id = 3d2078bd-1ba9-4423-ac3c-532f2e2601d2
5552025-03-25T23:00:24.675ZDEBGnexus (SEC): saga log event new_state = N006 started sec_id = 8f96aade-61ab-4710-a1b1-0efefa6704ee
5562025-03-25T23:00:24.675ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 6 saga_id = 3d2078bd-1ba9-4423-ac3c-532f2e2601d2
5572025-03-25T23:00:24.678ZDEBGnexus (SagaExecutor): instance update: committing new runtime state and unlocking... instance_id = 42a2133b-3721-4ce4-8ffe-ac478e6f81cd lock = UpdaterLock { updater_id: c073b03b-5408-437a-aecc-5a42e2f74e4e, locked_gen: Generation(Generation(3)) } new_runtime = InstanceRuntimeState { time_updated: 2025-03-25T23:00:24.439074494Z, gen: Generation(Generation(4)), propolis_id: None, dst_propolis_id: None, migration_id: None, nexus_state: NoVmm, time_last_auto_restarted: None } saga_id = 3d2078bd-1ba9-4423-ac3c-532f2e2601d2 saga_name = instance-update
5582025-03-25T23:00:24.678ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 3d2078bd-1ba9-4423-ac3c-532f2e2601d2 saga_name = instance-update saga_node = CommitInstanceUpdates
5592025-03-25T23:00:24.679ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 3d2078bd-1ba9-4423-ac3c-532f2e2601d2 saga_name = instance-update saga_node = CommitInstanceUpdates
5602025-03-25T23:00:24.739ZINFOnexus (SagaExecutor): instance update: committed update new runtime state! did_unlock = true file = nexus/src/app/sagas/instance_update/mod.rs:1176 instance_id = 42a2133b-3721-4ce4-8ffe-ac478e6f81cd new_runtime = InstanceRuntimeState { time_updated: 2025-03-25T23:00:24.439074494Z, gen: Generation(Generation(4)), propolis_id: None, dst_propolis_id: None, migration_id: None, nexus_state: NoVmm, time_last_auto_restarted: None } saga_id = 3d2078bd-1ba9-4423-ac3c-532f2e2601d2 saga_name = instance-update
5612025-03-25T23:00:24.739ZDEBGnexus (SEC): saga log event new_state = N006 succeeded sec_id = 8f96aade-61ab-4710-a1b1-0efefa6704ee
5622025-03-25T23:00:24.739ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 6 saga_id = 3d2078bd-1ba9-4423-ac3c-532f2e2601d2
5632025-03-25T23:00:24.739ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 23 reason = Signaled
5642025-03-25T23:00:24.739ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
5652025-03-25T23:00:24.739ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 23 reason = Signaled
5662025-03-25T23:00:24.739ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
5672025-03-25T23:00:24.739ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
5682025-03-25T23:00:24.739ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
5692025-03-25T23:00:24.739ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
5702025-03-25T23:00:24.741ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5712025-03-25T23:00:24.743ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
5722025-03-25T23:00:24.743ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
5732025-03-25T23:00:24.743ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
5742025-03-25T23:00:24.745ZDEBGnexus (SEC): saga log event new_state = N007 started sec_id = 8f96aade-61ab-4710-a1b1-0efefa6704ee
5752025-03-25T23:00:24.745ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 7 saga_id = 3d2078bd-1ba9-4423-ac3c-532f2e2601d2
5762025-03-25T23:00:24.745ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5772025-03-25T23:00:24.748ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
5782025-03-25T23:00:24.748ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
5792025-03-25T23:00:24.748ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
5802025-03-25T23:00:24.752ZDEBGnexus (SEC): saga log event new_state = N007 succeeded sec_id = 8f96aade-61ab-4710-a1b1-0efefa6704ee
5812025-03-25T23:00:24.752ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Object {"instance_id": String("42a2133b-3721-4ce4-8ffe-ac478e6f81cd"), "serialized_authn": Object {"kind": Object {"Authenticated": Array [Object {"actor": Object {"UserBuiltin": Object {"user_builtin_id": String("001de000-05e4-4000-8000-000000000002")}}}, Null]}}, "vmm_id": String("3214a533-86bc-409a-9c53-0114031452e2")}) node_id = 7 saga_id = 3d2078bd-1ba9-4423-ac3c-532f2e2601d2
5822025-03-25T23:00:24.756ZDEBGnexus (SEC): saga log event new_state = N008 started sec_id = 8f96aade-61ab-4710-a1b1-0efefa6704ee
5832025-03-25T23:00:24.756ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 8 saga_id = 3d2078bd-1ba9-4423-ac3c-532f2e2601d2
5842025-03-25T23:00:24.760ZDEBGnexus (SEC): saga log event new_state = N008 succeeded sec_id = 8f96aade-61ab-4710-a1b1-0efefa6704ee
5852025-03-25T23:00:24.760ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 8 saga_id = 3d2078bd-1ba9-4423-ac3c-532f2e2601d2
5862025-03-25T23:00:24.764ZDEBGnexus (SEC): saga log event new_state = N009 started sec_id = 8f96aade-61ab-4710-a1b1-0efefa6704ee
5872025-03-25T23:00:24.764ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 9 saga_id = 3d2078bd-1ba9-4423-ac3c-532f2e2601d2
5882025-03-25T23:00:24.767ZINFOnexus (SagaExecutor): instance update (VMM destroyed): deallocating sled resource reservation file = nexus/src/app/sagas/instance_update/destroyed.rs:82 instance_id = 42a2133b-3721-4ce4-8ffe-ac478e6f81cd propolis_id = 3214a533-86bc-409a-9c53-0114031452e2 saga_id = 3d2078bd-1ba9-4423-ac3c-532f2e2601d2 saga_name = instance-update
5892025-03-25T23:00:24.767ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 3d2078bd-1ba9-4423-ac3c-532f2e2601d2 saga_name = instance-update saga_node = ReleaseSledResources
5902025-03-25T23:00:24.767ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 3d2078bd-1ba9-4423-ac3c-532f2e2601d2 saga_name = instance-update saga_node = ReleaseSledResources
5912025-03-25T23:00:24.774ZDEBGnexus (SEC): saga log event new_state = N009 succeeded sec_id = 8f96aade-61ab-4710-a1b1-0efefa6704ee
5922025-03-25T23:00:24.774ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 9 saga_id = 3d2078bd-1ba9-4423-ac3c-532f2e2601d2
5932025-03-25T23:00:24.778ZDEBGnexus (SEC): saga log event new_state = N010 started sec_id = 8f96aade-61ab-4710-a1b1-0efefa6704ee
5942025-03-25T23:00:24.778ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 10 saga_id = 3d2078bd-1ba9-4423-ac3c-532f2e2601d2
5952025-03-25T23:00:24.782ZINFOnexus (SagaExecutor): instance update (VMM destroyed): marking VMM record deleted file = nexus/src/app/sagas/instance_update/destroyed.rs:113 instance_id = 42a2133b-3721-4ce4-8ffe-ac478e6f81cd propolis_id = 3214a533-86bc-409a-9c53-0114031452e2 saga_id = 3d2078bd-1ba9-4423-ac3c-532f2e2601d2 saga_name = instance-update
5962025-03-25T23:00:24.782ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 3d2078bd-1ba9-4423-ac3c-532f2e2601d2 saga_name = instance-update saga_node = MarkVmmDeleted
5972025-03-25T23:00:24.783ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 3d2078bd-1ba9-4423-ac3c-532f2e2601d2 saga_name = instance-update saga_node = MarkVmmDeleted
5982025-03-25T23:00:24.789ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
5992025-03-25T23:00:24.790ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
6002025-03-25T23:00:24.794ZDEBGnexus (SEC): saga log event new_state = N010 succeeded sec_id = 8f96aade-61ab-4710-a1b1-0efefa6704ee
6012025-03-25T23:00:24.794ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 10 saga_id = 3d2078bd-1ba9-4423-ac3c-532f2e2601d2
6022025-03-25T23:00:24.798ZDEBGnexus (SEC): saga log event new_state = N011 started sec_id = 8f96aade-61ab-4710-a1b1-0efefa6704ee
6032025-03-25T23:00:24.798ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 11 saga_id = 3d2078bd-1ba9-4423-ac3c-532f2e2601d2
6042025-03-25T23:00:24.801ZDEBGnexus (SEC): saga log event new_state = N011 succeeded sec_id = 8f96aade-61ab-4710-a1b1-0efefa6704ee
6052025-03-25T23:00:24.801ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 11 saga_id = 3d2078bd-1ba9-4423-ac3c-532f2e2601d2
6062025-03-25T23:00:24.803ZDEBGnexus (SEC): saga log event new_state = N012 started sec_id = 8f96aade-61ab-4710-a1b1-0efefa6704ee
6072025-03-25T23:00:24.803ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 12 saga_id = 3d2078bd-1ba9-4423-ac3c-532f2e2601d2
6082025-03-25T23:00:24.806ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 3d2078bd-1ba9-4423-ac3c-532f2e2601d2 saga_name = instance-update saga_node = ChainSuccessorSaga
6092025-03-25T23:00:24.807ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 3d2078bd-1ba9-4423-ac3c-532f2e2601d2 saga_name = instance-update saga_node = ChainSuccessorSaga
6102025-03-25T23:00:24.808ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 3d2078bd-1ba9-4423-ac3c-532f2e2601d2 saga_name = instance-update saga_node = ChainSuccessorSaga
6112025-03-25T23:00:24.809ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 3d2078bd-1ba9-4423-ac3c-532f2e2601d2 saga_name = instance-update saga_node = ChainSuccessorSaga
6122025-03-25T23:00:24.838ZINFOnexus (BackgroundTasks): VPC route manager sled 9af3e790-030d-4974-9b9e-c35f829f8643 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
6132025-03-25T23:00:24.838ZDEBGnexus (BackgroundTasks): client request SledAgent = 9af3e790-030d-4974-9b9e-c35f829f8643 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
6142025-03-25T23:00:24.839ZDEBGnexus (BackgroundTasks): client response SledAgent = 9af3e790-030d-4974-9b9e-c35f829f8643 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "3ad33c20-9ce8-4952-86c1-759358b2cc8d", "content-length": "372", "date": "Tue, 25 Mar 2025 23:00:24 GMT"} })
6152025-03-25T23:00:24.839ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
6162025-03-25T23:00:24.839ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
6172025-03-25T23:00:24.869ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 3d2078bd-1ba9-4423-ac3c-532f2e2601d2 saga_name = instance-update saga_node = ChainSuccessorSaga
6182025-03-25T23:00:24.869ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 3d2078bd-1ba9-4423-ac3c-532f2e2601d2 saga_name = instance-update saga_node = ChainSuccessorSaga
6192025-03-25T23:00:24.871ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } saga_id = 3d2078bd-1ba9-4423-ac3c-532f2e2601d2 saga_name = instance-update saga_node = ChainSuccessorSaga
6202025-03-25T23:00:24.874ZDEBGnexus (SagaExecutor): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Instance { parent: Project { parent: Silo { parent: Fleet, key: 526efdec-95ea-41d0-8852-ffb890a1c1a5, lookup_type: ById(526efdec-95ea-41d0-8852-ffb890a1c1a5) }, key: 2fdb0ee2-76b2-4482-8062-671d3c8196a5, lookup_type: ById(2fdb0ee2-76b2-4482-8062-671d3c8196a5) }, key: 42a2133b-3721-4ce4-8ffe-ac478e6f81cd, lookup_type: ById(42a2133b-3721-4ce4-8ffe-ac478e6f81cd) } result = Ok(()) saga_id = 3d2078bd-1ba9-4423-ac3c-532f2e2601d2 saga_name = instance-update saga_node = ChainSuccessorSaga
6212025-03-25T23:00:24.874ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 3d2078bd-1ba9-4423-ac3c-532f2e2601d2 saga_name = instance-update saga_node = ChainSuccessorSaga
6222025-03-25T23:00:24.874ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 3d2078bd-1ba9-4423-ac3c-532f2e2601d2 saga_name = instance-update saga_node = ChainSuccessorSaga
6232025-03-25T23:00:24.877ZDEBGnexus (SagaExecutor): instance update: instance will not reincarnate auto_restart_config = InstanceAutoRestart { policy: None, cooldown: None } instance_id = 42a2133b-3721-4ce4-8ffe-ac478e6f81cd karmic_state = WillReincarnate needs_reincarnation = false saga_id = 3d2078bd-1ba9-4423-ac3c-532f2e2601d2 saga_name = instance-update
6242025-03-25T23:00:24.877ZDEBGnexus (SEC): saga log event new_state = N012 succeeded sec_id = 8f96aade-61ab-4710-a1b1-0efefa6704ee
6252025-03-25T23:00:24.877ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 12 saga_id = 3d2078bd-1ba9-4423-ac3c-532f2e2601d2
6262025-03-25T23:00:24.881ZDEBGnexus (SEC): saga log event new_state = N014 started sec_id = 8f96aade-61ab-4710-a1b1-0efefa6704ee
6272025-03-25T23:00:24.881ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 14 saga_id = 3d2078bd-1ba9-4423-ac3c-532f2e2601d2
6282025-03-25T23:00:24.882ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
6292025-03-25T23:00:24.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
6302025-03-25T23:00:24.883ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = buskin
6312025-03-25T23:00:24.883ZDEBGnexus (BackgroundTasks): client request SledAgent = 9af3e790-030d-4974-9b9e-c35f829f8643 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
6322025-03-25T23:00:24.883ZDEBGnexus (BackgroundTasks): client response SledAgent = 9af3e790-030d-4974-9b9e-c35f829f8643 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "57acda20-dd9f-4e07-a6b9-bd8e3a149e67", "date": "Tue, 25 Mar 2025 23:00:24 GMT"} })
6332025-03-25T23:00:24.883ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
6342025-03-25T23:00:24.884ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
6352025-03-25T23:00:24.884ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6362025-03-25T23:00:24.884ZDEBGnexus (SEC): saga log event new_state = N014 succeeded sec_id = 8f96aade-61ab-4710-a1b1-0efefa6704ee
6372025-03-25T23:00:24.884ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 14 saga_id = 3d2078bd-1ba9-4423-ac3c-532f2e2601d2
6382025-03-25T23:00:24.886ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
6392025-03-25T23:00:24.886ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
6402025-03-25T23:00:24.886ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
6412025-03-25T23:00:24.886ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
6422025-03-25T23:00:24.886ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
6432025-03-25T23:00:24.888ZINFOnexus (SEC): update for saga cached state file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/steno-0.4.1/src/sec.rs:1384 new_state = Done saga_id = 3d2078bd-1ba9-4423-ac3c-532f2e2601d2 sec_id = 8f96aade-61ab-4710-a1b1-0efefa6704ee
6442025-03-25T23:00:24.888ZINFOnexus (SecStore): updating state file = nexus/db-queries/src/db/sec_store.rs:95 new_state = done saga_id = 3d2078bd-1ba9-4423-ac3c-532f2e2601d2
6452025-03-25T23:00:24.888ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6462025-03-25T23:00:24.889ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
6472025-03-25T23:00:24.889ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
6482025-03-25T23:00:24.890ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
6492025-03-25T23:00:24.890ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
6502025-03-25T23:00:24.890ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
6512025-03-25T23:00:24.892ZINFOnexus (BackgroundTasks): VPC route manager sled 9af3e790-030d-4974-9b9e-c35f829f8643 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
6522025-03-25T23:00:24.892ZDEBGnexus (BackgroundTasks): client request SledAgent = 9af3e790-030d-4974-9b9e-c35f829f8643 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
6532025-03-25T23:00:24.892ZDEBGnexus (BackgroundTasks): client response SledAgent = 9af3e790-030d-4974-9b9e-c35f829f8643 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "f3bd4849-4d74-467f-adc2-ec910bf7aa3d", "date": "Tue, 25 Mar 2025 23:00:24 GMT"} })
6542025-03-25T23:00:24.894ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 154.956923ms iteration = 23 status = {}
6552025-03-25T23:00:24.926ZINFOnexus (SEC): saga finished file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/steno-0.4.1/src/sec.rs:1025 result = success saga_id = 3d2078bd-1ba9-4423-ac3c-532f2e2601d2 saga_name = instance-update sec_id = 8f96aade-61ab-4710-a1b1-0efefa6704ee
6562025-03-25T23:00:24.928ZINFOnexus (SagaExecutor): saga finished file = nexus/src/app/saga.rs:404 saga_id = 3d2078bd-1ba9-4423-ac3c-532f2e2601d2 saga_name = instance-update saga_result = SagaResult { saga_id: 3d2078bd-1ba9-4423-ac3c-532f2e2601d2, saga_log: SagaLog { saga_id: 3d2078bd-1ba9-4423-ac3c-532f2e2601d2, unwinding: false, events: [N013 started, N013 succeeded, N000 started, N000 succeeded, N001 started, N001 succeeded, N002 started, N002 succeeded, N003 started, N003 succeeded, N004 started, N004 succeeded, N005 started, N005 succeeded, N006 started, N006 succeeded, N007 started, N007 succeeded, N008 started, N008 succeeded, N009 started, N009 succeeded, N010 started, N010 succeeded, N011 started, N011 succeeded, N012 started, N012 succeeded, N014 started, N014 succeeded], node_status: {0: Succeeded(String("c073b03b-5408-437a-aecc-5a42e2f74e4e")), 1: Succeeded(Object {"locked_gen": Number(3), "updater_id": String("c073b03b-5408-437a-aecc-5a42e2f74e4e")}), 2: Succeeded(String("Delete")), 3: Succeeded(Null), 4: Succeeded(Null), 5: Succeeded(Null), 6: Succeeded(Null), 7: Succeeded(Object {"instance_id": String("42a2133b-3721-4ce4-8ffe-ac478e6f81cd"), "serialized_authn": Object {"kind": Object {"Authenticated": Array [Object {"actor": Object {"UserBuiltin": Object {"user_builtin_id": String("001de000-05e4-4000-8000-000000000002")}}}, Null]}}, "vmm_id": String("3214a533-86bc-409a-9c53-0114031452e2")}), 8: Succeeded(Null), 9: Succeeded(Null), 10: Succeeded(Null), 11: Succeeded(Null), 12: Succeeded(Null), 13: Succeeded(Null), 14: Succeeded(Null)} }, kind: Ok(SagaResultOk { saga_output: Null, node_outputs: {"active_vmm_destroyed": Null, "chain_successor_saga": Null, "commit_instance_updates": Null, "destroy_active_vmm_params": Object {"instance_id": String("42a2133b-3721-4ce4-8ffe-ac478e6f81cd"), "serialized_authn": Object {"kind": Object {"Authenticated": Array [Object {"actor": Object {"UserBuiltin": Object {"user_builtin_id": String("001de000-05e4-4000-8000-000000000002")}}}, Null]}}, "vmm_id": String("3214a533-86bc-409a-9c53-0114031452e2")}, "mark_vmm_deleted": Null, "network_config_update": String("Delete"), "release_sled_resources": Null, "release_virtual_provisioning": Null, "saga_instance_lock_id": String("c073b03b-5408-437a-aecc-5a42e2f74e4e"), "unassign_oximeter_producer": Null, "update_network_config": Null, "updater_lock": Object {"locked_gen": Number(3), "updater_id": String("c073b03b-5408-437a-aecc-5a42e2f74e4e")}} }) }
6572025-03-25T23:00:24.928ZDEBGnexus (SagaExecutor): instance update: child saga completed successfully child_saga_id = 3d2078bd-1ba9-4423-ac3c-532f2e2601d2 instance_id = 42a2133b-3721-4ce4-8ffe-ac478e6f81cd saga_id = 1d5c0462-def7-4d55-8913-c2e4e45419a6 saga_name = start-instance-update
6582025-03-25T23:00:24.928ZDEBGnexus (SEC): saga log event new_state = N002 succeeded sec_id = 8f96aade-61ab-4710-a1b1-0efefa6704ee
6592025-03-25T23:00:24.928ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 2 saga_id = 1d5c0462-def7-4d55-8913-c2e4e45419a6
6602025-03-25T23:00:24.932ZDEBGnexus (SEC): saga log event new_state = N004 started sec_id = 8f96aade-61ab-4710-a1b1-0efefa6704ee
6612025-03-25T23:00:24.932ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 4 saga_id = 1d5c0462-def7-4d55-8913-c2e4e45419a6
6622025-03-25T23:00:24.935ZDEBGnexus (SEC): saga log event new_state = N004 succeeded sec_id = 8f96aade-61ab-4710-a1b1-0efefa6704ee
6632025-03-25T23:00:24.935ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 4 saga_id = 1d5c0462-def7-4d55-8913-c2e4e45419a6
6642025-03-25T23:00:24.938ZINFOnexus (SEC): update for saga cached state file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/steno-0.4.1/src/sec.rs:1384 new_state = Done saga_id = 1d5c0462-def7-4d55-8913-c2e4e45419a6 sec_id = 8f96aade-61ab-4710-a1b1-0efefa6704ee
6652025-03-25T23:00:24.938ZINFOnexus (SecStore): updating state file = nexus/db-queries/src/db/sec_store.rs:95 new_state = done saga_id = 1d5c0462-def7-4d55-8913-c2e4e45419a6
6662025-03-25T23:00:24.945ZINFOnexus (SEC): saga finished file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/steno-0.4.1/src/sec.rs:1025 result = success saga_id = 1d5c0462-def7-4d55-8913-c2e4e45419a6 saga_name = start-instance-update sec_id = 8f96aade-61ab-4710-a1b1-0efefa6704ee
6672025-03-25T23:00:24.945ZINFOnexus (SagaExecutor): saga finished file = nexus/src/app/saga.rs:404 saga_id = 1d5c0462-def7-4d55-8913-c2e4e45419a6 saga_name = start-instance-update saga_result = SagaResult { saga_id: 1d5c0462-def7-4d55-8913-c2e4e45419a6, saga_log: SagaLog { saga_id: 1d5c0462-def7-4d55-8913-c2e4e45419a6, unwinding: false, events: [N003 started, N003 succeeded, N000 started, N000 succeeded, N001 started, N001 succeeded, N002 started, N002 succeeded, N004 started, N004 succeeded], node_status: {0: Succeeded(String("865e2d38-4548-454c-b8f3-af8630bf45fc")), 1: Succeeded(Object {"locked_gen": Number(2), "updater_id": String("865e2d38-4548-454c-b8f3-af8630bf45fc")}), 2: Succeeded(Null), 3: Succeeded(Null), 4: Succeeded(Null)} }, kind: Ok(SagaResultOk { saga_output: Null, node_outputs: {"saga_instance_lock_id": String("865e2d38-4548-454c-b8f3-af8630bf45fc"), "state": Null, "updater_lock": Object {"locked_gen": Number(2), "updater_id": String("865e2d38-4548-454c-b8f3-af8630bf45fc")}} }) }
6682025-03-25T23:00:24.979ZDEBGnexus (BackgroundTasks): client request SledAgent = 9af3e790-030d-4974-9b9e-c35f829f8643 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
6692025-03-25T23:00:24.980ZDEBGnexus (BackgroundTasks): client response SledAgent = 9af3e790-030d-4974-9b9e-c35f829f8643 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "bcc91fc7-4d94-46a1-abf1-adb34f4244af", "content-length": "691", "date": "Tue, 25 Mar 2025 23:00:24 GMT"} })
6702025-03-25T23:00:24.980ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = buskin
6712025-03-25T23:00:24.980ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = buskin
6722025-03-25T23:00:24.980ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = buskin
6732025-03-25T23:00:24.982ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 242.933166ms iteration = 23 status = {}
6742025-03-25T23:00:24.990ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 624d4ec1-8b92-4f6c-ae4a-189491df8d88 roles = RoleSet { roles: {} } uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
6752025-03-25T23:00:24.991ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 624d4ec1-8b92-4f6c-ae4a-189491df8d88 resource = Database result = Ok(()) uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
6762025-03-25T23:00:24.992ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1938 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 624d4ec1-8b92-4f6c-ae4a-189491df8d88 response_code = 200 uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
6772025-03-25T23:00:26.013ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 355f7099-e17c-4ce4-b430-bf28b896de6d roles = RoleSet { roles: {} } uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
6782025-03-25T23:00:26.014ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 355f7099-e17c-4ce4-b430-bf28b896de6d resource = Database result = Ok(()) uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
6792025-03-25T23:00:26.015ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1780 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 355f7099-e17c-4ce4-b430-bf28b896de6d response_code = 200 uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
6802025-03-25T23:00:27.036ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 1a959650-0304-41be-9cca-282858428c6d roles = RoleSet { roles: {} } uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
6812025-03-25T23:00:27.037ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 1a959650-0304-41be-9cca-282858428c6d resource = Database result = Ok(()) uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
6822025-03-25T23:00:27.038ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1984 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 1a959650-0304-41be-9cca-282858428c6d response_code = 200 uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
6832025-03-25T23:00:28.060ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 6f1f4ccd-a6fe-4b5c-9b1e-03ca19a7f6df roles = RoleSet { roles: {} } uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
6842025-03-25T23:00:28.061ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 6f1f4ccd-a6fe-4b5c-9b1e-03ca19a7f6df resource = Database result = Ok(()) uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
6852025-03-25T23:00:28.063ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2415 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 6f1f4ccd-a6fe-4b5c-9b1e-03ca19a7f6df response_code = 200 uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
6862025-03-25T23:00:29.083ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 9373ac1b-1e4a-4af7-9ce7-ca43fd65f181 roles = RoleSet { roles: {} } uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
6872025-03-25T23:00:29.083ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 9373ac1b-1e4a-4af7-9ce7-ca43fd65f181 resource = Database result = Ok(()) uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
6882025-03-25T23:00:29.085ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2625 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 9373ac1b-1e4a-4af7-9ce7-ca43fd65f181 response_code = 200 uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
6892025-03-25T23:00:30.105ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 7f03b8a0-cdd0-4652-843a-22bc07dd746a roles = RoleSet { roles: {} } uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
6902025-03-25T23:00:30.105ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 7f03b8a0-cdd0-4652-843a-22bc07dd746a resource = Database result = Ok(()) uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
6912025-03-25T23:00:30.107ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2449 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 7f03b8a0-cdd0-4652-843a-22bc07dd746a response_code = 200 uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
6922025-03-25T23:00:31.127ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 75d9b248-8bce-4138-a1a7-b530a51d2d76 roles = RoleSet { roles: {} } uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
6932025-03-25T23:00:31.128ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 75d9b248-8bce-4138-a1a7-b530a51d2d76 resource = Database result = Ok(()) uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
6942025-03-25T23:00:31.130ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2441 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 75d9b248-8bce-4138-a1a7-b530a51d2d76 response_code = 200 uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
6952025-03-25T23:00:31.996ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 66 reason = Timeout
6962025-03-25T23:00:31.996ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6972025-03-25T23:00:31.996ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6982025-03-25T23:00:31.996ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 66 reason = Timeout
6992025-03-25T23:00:31.996ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
7002025-03-25T23:00:31.997ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
7012025-03-25T23:00:31.998ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7022025-03-25T23:00:32.001ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
7032025-03-25T23:00:32.001ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
7042025-03-25T23:00:32.001ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
7052025-03-25T23:00:32.004ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
7062025-03-25T23:00:32.004ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
7072025-03-25T23:00:32.005ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7082025-03-25T23:00:32.008ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 5d9d6f43-fd71-4256-b06d-1ed4c87e191f, lookup_type: ById(5d9d6f43-fd71-4256-b06d-1ed4c87e191f) } result = Ok(())
7092025-03-25T23:00:32.008ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
7102025-03-25T23:00:32.008ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
7112025-03-25T23:00:32.019ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 5d9d6f43-fd71-4256-b06d-1ed4c87e191f original_time_created = 2025-03-25 22:49:35.383803 UTC target_id = 5d9d6f43-fd71-4256-b06d-1ed4c87e191f time_created = 2025-03-25 22:49:35.383803 UTC
7122025-03-25T23:00:32.019ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 23.053549ms iteration = 66 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"5d9d6f43-fd71-4256-b06d-1ed4c87e191f","time_created":"2025-03-25T22:49:35.383803Z"}
7132025-03-25T23:00:32.030ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
7142025-03-25T23:00:32.030ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
7152025-03-25T23:00:32.059ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 62.220593ms iteration = 66 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
7162025-03-25T23:00:32.150ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = d29cb5f6-dd75-4865-9e59-e3e72045c22c roles = RoleSet { roles: {} } uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
7172025-03-25T23:00:32.150ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = d29cb5f6-dd75-4865-9e59-e3e72045c22c resource = Database result = Ok(()) uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
7182025-03-25T23:00:32.152ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1913 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = d29cb5f6-dd75-4865-9e59-e3e72045c22c response_code = 200 uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
7192025-03-25T23:00:33.173ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 6c4016dc-e66f-476e-bfb6-ae0e450a941e roles = RoleSet { roles: {} } uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
7202025-03-25T23:00:33.174ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 6c4016dc-e66f-476e-bfb6-ae0e450a941e resource = Database result = Ok(()) uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
7212025-03-25T23:00:33.175ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2122 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 6c4016dc-e66f-476e-bfb6-ae0e450a941e response_code = 200 uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
7222025-03-25T23:00:33.856ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 731 local_addr = [fd00:1122:3344:101::a]:62174 method = GET remote_addr = [fd00:1122:3344:101::d]:63115 req_id = d7ef8d2c-6645-4847-ae68-e89f7859883b response_code = 200 uri = /8f96aade-61ab-4710-a1b1-0efefa6704ee
7232025-03-25T23:00:34.197ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 7b30b557-db4e-45d8-8c25-5855b34af8aa roles = RoleSet { roles: {} } uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
7242025-03-25T23:00:34.197ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 7b30b557-db4e-45d8-8c25-5855b34af8aa resource = Database result = Ok(()) uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
7252025-03-25T23:00:34.199ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2598 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 7b30b557-db4e-45d8-8c25-5855b34af8aa response_code = 200 uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
7262025-03-25T23:00:35.220ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 980ce53f-27cc-4251-a257-af68834c2ddc roles = RoleSet { roles: {} } uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
7272025-03-25T23:00:35.220ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 980ce53f-27cc-4251-a257-af68834c2ddc resource = Database result = Ok(()) uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
7282025-03-25T23:00:35.222ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2340 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 980ce53f-27cc-4251-a257-af68834c2ddc response_code = 200 uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
7292025-03-25T23:00:36.243ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 30f60909-f89b-46c1-9a84-8268a4b9f51d roles = RoleSet { roles: {} } uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
7302025-03-25T23:00:36.243ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 30f60909-f89b-46c1-9a84-8268a4b9f51d resource = Database result = Ok(()) uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
7312025-03-25T23:00:36.245ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2297 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 30f60909-f89b-46c1-9a84-8268a4b9f51d response_code = 200 uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
7322025-03-25T23:00:37.266ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = fc27ca27-7bdf-4369-b6a3-c408b0b35c73 roles = RoleSet { roles: {} } uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
7332025-03-25T23:00:37.267ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = fc27ca27-7bdf-4369-b6a3-c408b0b35c73 resource = Database result = Ok(()) uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
7342025-03-25T23:00:37.269ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2411 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = fc27ca27-7bdf-4369-b6a3-c408b0b35c73 response_code = 200 uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
7352025-03-25T23:00:38.289ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = afbb3ab6-64b4-45a5-a1d9-f0d9ad60a810 roles = RoleSet { roles: {} } uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
7362025-03-25T23:00:38.290ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = afbb3ab6-64b4-45a5-a1d9-f0d9ad60a810 resource = Database result = Ok(()) uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
7372025-03-25T23:00:38.292ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 3013 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = afbb3ab6-64b4-45a5-a1d9-f0d9ad60a810 response_code = 200 uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
7382025-03-25T23:00:39.313ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 019f8be2-98bd-4fd4-9c85-e57125331757 roles = RoleSet { roles: {} } uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
7392025-03-25T23:00:39.314ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 019f8be2-98bd-4fd4-9c85-e57125331757 resource = Database result = Ok(()) uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
7402025-03-25T23:00:39.315ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2266 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 019f8be2-98bd-4fd4-9c85-e57125331757 response_code = 200 uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
7412025-03-25T23:00:40.335ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 32389b4b-f8c0-4d87-af37-3a50216a653a roles = RoleSet { roles: {} } uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
7422025-03-25T23:00:40.335ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 32389b4b-f8c0-4d87-af37-3a50216a653a resource = Database result = Ok(()) uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
7432025-03-25T23:00:40.337ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1861 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 32389b4b-f8c0-4d87-af37-3a50216a653a response_code = 200 uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
7442025-03-25T23:00:41.357ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 643e4319-05a9-48f2-92d1-3c871eba2dc4 roles = RoleSet { roles: {} } uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
7452025-03-25T23:00:41.358ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 643e4319-05a9-48f2-92d1-3c871eba2dc4 resource = Database result = Ok(()) uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
7462025-03-25T23:00:41.360ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 3029 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 643e4319-05a9-48f2-92d1-3c871eba2dc4 response_code = 200 uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
7472025-03-25T23:00:41.995ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 67 reason = Timeout
7482025-03-25T23:00:41.995ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 12 reason = Timeout
7492025-03-25T23:00:41.995ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
7502025-03-25T23:00:41.995ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 12 reason = Timeout
7512025-03-25T23:00:41.995ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 14 reason = Timeout
7522025-03-25T23:00:41.995ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
7532025-03-25T23:00:41.995ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 12 reason = Timeout
7542025-03-25T23:00:41.995ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 14 reason = Timeout
7552025-03-25T23:00:41.995ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
7562025-03-25T23:00:41.995ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
7572025-03-25T23:00:41.995ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 12 reason = Timeout
7582025-03-25T23:00:41.995ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2025-03-25 22:50:41.995833986 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
7592025-03-25T23:00:41.995ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 23 reason = Timeout
7602025-03-25T23:00:41.995ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
7612025-03-25T23:00:41.995ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
7622025-03-25T23:00:41.995ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 23 reason = Timeout
7632025-03-25T23:00:41.995ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
7642025-03-25T23:00:41.995ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 12 reason = Timeout
7652025-03-25T23:00:41.995ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
7662025-03-25T23:00:41.995ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
7672025-03-25T23:00:41.995ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 23 reason = Timeout
7682025-03-25T23:00:41.995ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 13 reason = Timeout
7692025-03-25T23:00:41.995ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:82 reason = blueprint disabled target_id = 5d9d6f43-fd71-4256-b06d-1ed4c87e191f
7702025-03-25T23:00:41.995ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 12 reason = Timeout
7712025-03-25T23:00:41.996ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 15.01\u{b5}s iteration = 13 status = {"enabled":false,"target_id":"5d9d6f43-fd71-4256-b06d-1ed4c87e191f"}
7722025-03-25T23:00:41.996ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
7732025-03-25T23:00:41.996ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
7742025-03-25T23:00:41.996ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
7752025-03-25T23:00:41.996ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
7762025-03-25T23:00:41.996ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
7772025-03-25T23:00:41.996ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
7782025-03-25T23:00:41.996ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
7792025-03-25T23:00:41.996ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("3d421061-bd04-4ffe-bd2d-2457b6327712.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("5b0364ab-2148-452d-a2c6-c3386f36f703.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("5b0364ab-2148-452d-a2c6-c3386f36f703.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::8))) }], valid_until: Instant { tv_sec: 1815, tv_nsec: 442357602 } })
7802025-03-25T23:00:41.996ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("1f1d5f3e-c8d2-4c62-bebc-2b91d1b5029d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("25bc856c-a30e-4fd6-94c7-e4c386b38177.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("eb0f1e78-4f86-4f9f-bbd8-3669a0f1cb98.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("eb0f1e78-4f86-4f9f-bbd8-3669a0f1cb98.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:1::1))) }], valid_until: Instant { tv_sec: 1815, tv_nsec: 442449484 } })
7812025-03-25T23:00:41.996ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 23 reason = Timeout
7822025-03-25T23:00:41.996ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 26 reason = Timeout
7832025-03-25T23:00:41.996ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 23 reason = Timeout
7842025-03-25T23:00:41.996ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
7852025-03-25T23:00:41.996ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
7862025-03-25T23:00:41.996ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
7872025-03-25T23:00:41.996ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
7882025-03-25T23:00:41.996ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 12 reason = Timeout
7892025-03-25T23:00:41.996ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
7902025-03-25T23:00:41.996ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 23 reason = Timeout
7912025-03-25T23:00:41.996ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
7922025-03-25T23:00:41.996ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 67 reason = Timeout
7932025-03-25T23:00:41.996ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 23 reason = Timeout
7942025-03-25T23:00:41.996ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
7952025-03-25T23:00:41.996ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
7962025-03-25T23:00:41.996ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 23 reason = Timeout
7972025-03-25T23:00:41.996ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
7982025-03-25T23:00:41.996ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 12 reason = Timeout
7992025-03-25T23:00:41.996ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:56
8002025-03-25T23:00:41.996ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 23 reason = Timeout
8012025-03-25T23:00:41.996ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
8022025-03-25T23:00:41.996ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
8032025-03-25T23:00:41.996ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 23 reason = Timeout
8042025-03-25T23:00:41.996ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
8052025-03-25T23:00:41.996ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 23 reason = Timeout
8062025-03-25T23:00:41.996ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
8072025-03-25T23:00:41.996ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 23 reason = Timeout
8082025-03-25T23:00:41.996ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
8092025-03-25T23:00:41.996ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 24 reason = Timeout
8102025-03-25T23:00:41.996ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 12 reason = Timeout
8112025-03-25T23:00:41.996ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
8122025-03-25T23:00:41.996ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
8132025-03-25T23:00:41.996ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 23 reason = Timeout
8142025-03-25T23:00:41.996ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
8152025-03-25T23:00:41.996ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 23 reason = Timeout
8162025-03-25T23:00:41.996ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
8172025-03-25T23:00:41.996ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 24 reason = Timeout
8182025-03-25T23:00:41.996ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
8192025-03-25T23:00:41.996ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
8202025-03-25T23:00:41.996ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 12 reason = Timeout
8212025-03-25T23:00:41.996ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
8222025-03-25T23:00:41.996ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
8232025-03-25T23:00:41.996ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
8242025-03-25T23:00:41.997ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
8252025-03-25T23:00:41.997ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
8262025-03-25T23:00:41.997ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } dns_group = external
8272025-03-25T23:00:41.997ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.288189ms iteration = 12 status = {"addresses":["[fd00:1122:3344:101::9]:5353","[fd00:1122:3344:101::8]:5353"]}
8282025-03-25T23:00:41.997ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
8292025-03-25T23:00:41.997ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
8302025-03-25T23:00:41.997ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
8312025-03-25T23:00:41.997ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
8322025-03-25T23:00:41.997ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
8332025-03-25T23:00:41.997ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
8342025-03-25T23:00:41.997ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
8352025-03-25T23:00:41.997ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
8362025-03-25T23:00:41.997ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
8372025-03-25T23:00:41.997ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] } dns_group = internal
8382025-03-25T23:00:41.997ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.363761ms iteration = 12 status = {"addresses":["[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:3::1]:5353","[fd00:1122:3344:1::1]:5353"]}
8392025-03-25T23:00:41.997ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
8402025-03-25T23:00:41.997ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
8412025-03-25T23:00:41.997ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
8422025-03-25T23:00:41.997ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
8432025-03-25T23:00:41.997ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
8442025-03-25T23:00:41.997ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
8452025-03-25T23:00:41.998ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8462025-03-25T23:00:41.999ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
8472025-03-25T23:00:41.999ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.319215ms iteration = 23 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
8482025-03-25T23:00:41.999ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
8492025-03-25T23:00:41.999ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2025-03-25 22:50:41.995833986 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
8502025-03-25T23:00:41.999ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 3.630122ms iteration = 12 status = {"expiration":"2025-03-25T22:50:41.995833986Z","pruned":[]}
8512025-03-25T23:00:42.000ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-9af3e790-030d-4974-9b9e-c35f829f8643.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-9af3e790-030d-4974-9b9e-c35f829f8643.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1815, tv_nsec: 446002294 } })
8522025-03-25T23:00:42.000ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8532025-03-25T23:00:42.000ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1077
8542025-03-25T23:00:42.001ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
8552025-03-25T23:00:42.001ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
8562025-03-25T23:00:42.002ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
8572025-03-25T23:00:42.002ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
8582025-03-25T23:00:42.002ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
8592025-03-25T23:00:42.002ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
8602025-03-25T23:00:42.002ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
8612025-03-25T23:00:42.002ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
8622025-03-25T23:00:42.004ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-9af3e790-030d-4974-9b9e-c35f829f8643.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-9af3e790-030d-4974-9b9e-c35f829f8643.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1815, tv_nsec: 450573587 } })
8632025-03-25T23:00:42.004ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1077
8642025-03-25T23:00:42.004ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8652025-03-25T23:00:42.006ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
8662025-03-25T23:00:42.007ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
8672025-03-25T23:00:42.007ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
8682025-03-25T23:00:42.008ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
8692025-03-25T23:00:42.008ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
8702025-03-25T23:00:42.010ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8712025-03-25T23:00:42.013ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
8722025-03-25T23:00:42.013ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
8732025-03-25T23:00:42.013ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
8742025-03-25T23:00:42.020ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 23.778316ms iteration = 12 status = {"errors":[],"found_port_ok":[]}
8752025-03-25T23:00:42.022ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
8762025-03-25T23:00:42.022ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8772025-03-25T23:00:42.023ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
8782025-03-25T23:00:42.023ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
8792025-03-25T23:00:42.024ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
8802025-03-25T23:00:42.027ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
8812025-03-25T23:00:42.028ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8822025-03-25T23:00:42.030ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
8832025-03-25T23:00:42.030ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
8842025-03-25T23:00:42.032ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
8852025-03-25T23:00:42.033ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
8862025-03-25T23:00:42.034ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 37.674909ms iteration = 23 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
8872025-03-25T23:00:42.034ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
8882025-03-25T23:00:42.036ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8892025-03-25T23:00:42.040ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 5d9d6f43-fd71-4256-b06d-1ed4c87e191f, lookup_type: ById(5d9d6f43-fd71-4256-b06d-1ed4c87e191f) } result = Ok(())
8902025-03-25T23:00:42.040ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
8912025-03-25T23:00:42.040ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
8922025-03-25T23:00:42.042ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8932025-03-25T23:00:42.044ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 47.48738ms iteration = 23 status = {"errors":[],"requests_created_ok":[]}
8942025-03-25T23:00:42.044ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8952025-03-25T23:00:42.045ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 49.112328ms iteration = 23 status = {"errors":[],"garbage_collect_requested":[]}
8962025-03-25T23:00:42.046ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
8972025-03-25T23:00:42.046ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
8982025-03-25T23:00:42.046ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
8992025-03-25T23:00:42.046ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 49.674219ms iteration = 23 status = {"errors":[],"finish_invoked_ok":[]}
9002025-03-25T23:00:42.047ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
9012025-03-25T23:00:42.047ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
9022025-03-25T23:00:42.048ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
9032025-03-25T23:00:42.048ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
9042025-03-25T23:00:42.048ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9052025-03-25T23:00:42.049ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
9062025-03-25T23:00:42.049ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9072025-03-25T23:00:42.050ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9082025-03-25T23:00:42.051ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
9092025-03-25T23:00:42.051ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
9102025-03-25T23:00:42.051ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
9112025-03-25T23:00:42.051ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
9122025-03-25T23:00:42.051ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
9132025-03-25T23:00:42.051ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
9142025-03-25T23:00:42.051ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
9152025-03-25T23:00:42.052ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
9162025-03-25T23:00:42.052ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:167 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
9172025-03-25T23:00:42.052ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 55.367229ms iteration = 12 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
9182025-03-25T23:00:42.052ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
9192025-03-25T23:00:42.052ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
9202025-03-25T23:00:42.052ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
9212025-03-25T23:00:42.052ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
9222025-03-25T23:00:42.052ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
9232025-03-25T23:00:42.052ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
9242025-03-25T23:00:42.053ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9252025-03-25T23:00:42.054ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 58.082079ms iteration = 67 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
9262025-03-25T23:00:42.056ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
9272025-03-25T23:00:42.056ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
9282025-03-25T23:00:42.056ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-9af3e790-030d-4974-9b9e-c35f829f8643.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-9af3e790-030d-4974-9b9e-c35f829f8643.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1815, tv_nsec: 502655861 } })
9292025-03-25T23:00:42.056ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
9302025-03-25T23:00:42.057ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1077 rack_id = 3ef3cc6b-3ffd-47a3-99ef-c57b25776619
9312025-03-25T23:00:42.060ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9322025-03-25T23:00:42.060ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
9332025-03-25T23:00:42.061ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
9342025-03-25T23:00:42.061ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
9352025-03-25T23:00:42.061ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
9362025-03-25T23:00:42.061ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
9372025-03-25T23:00:42.061ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
9382025-03-25T23:00:42.061ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
9392025-03-25T23:00:42.061ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9402025-03-25T23:00:42.063ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
9412025-03-25T23:00:42.063ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
9422025-03-25T23:00:42.063ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
9432025-03-25T23:00:42.064ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
9442025-03-25T23:00:42.064ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
9452025-03-25T23:00:42.065ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
9462025-03-25T23:00:42.067ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9472025-03-25T23:00:42.071ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
9482025-03-25T23:00:42.071ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
9492025-03-25T23:00:42.071ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
9502025-03-25T23:00:42.074ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
9512025-03-25T23:00:42.074ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
9522025-03-25T23:00:42.076ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 5d9d6f43-fd71-4256-b06d-1ed4c87e191f original_time_created = 2025-03-25 22:49:35.383803 UTC target_id = 5d9d6f43-fd71-4256-b06d-1ed4c87e191f time_created = 2025-03-25 22:49:35.383803 UTC
9532025-03-25T23:00:42.076ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 80.864942ms iteration = 67 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"5d9d6f43-fd71-4256-b06d-1ed4c87e191f","time_created":"2025-03-25T22:49:35.383803Z"}
9542025-03-25T23:00:42.077ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 1 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
9552025-03-25T23:00:42.077ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 80.549445ms iteration = 23 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":1,"total_instances":0,"update_sagas_queued":0}
9562025-03-25T23:00:42.087ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
9572025-03-25T23:00:42.087ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
9582025-03-25T23:00:42.087ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
9592025-03-25T23:00:42.087ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
9602025-03-25T23:00:42.088ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
9612025-03-25T23:00:42.089ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9622025-03-25T23:00:42.094ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
9632025-03-25T23:00:42.094ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
9642025-03-25T23:00:42.094ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
9652025-03-25T23:00:42.094ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
9662025-03-25T23:00:42.095ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
9672025-03-25T23:00:42.097ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
9682025-03-25T23:00:42.097ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
9692025-03-25T23:00:42.097ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
9702025-03-25T23:00:42.097ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 39030d55-287b-484a-98ee-50f0b968e055 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
9712025-03-25T23:00:42.097ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
9722025-03-25T23:00:42.098ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
9732025-03-25T23:00:42.098ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9742025-03-25T23:00:42.099ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9752025-03-25T23:00:42.100ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
9762025-03-25T23:00:42.101ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
9772025-03-25T23:00:42.102ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 9af3e790-030d-4974-9b9e-c35f829f8643, lookup_type: ById(9af3e790-030d-4974-9b9e-c35f829f8643) } result = Ok(())
9782025-03-25T23:00:42.102ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
9792025-03-25T23:00:42.102ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
9802025-03-25T23:00:42.102ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
9812025-03-25T23:00:42.102ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
9822025-03-25T23:00:42.103ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
9832025-03-25T23:00:42.104ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
9842025-03-25T23:00:42.104ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
9852025-03-25T23:00:42.104ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
9862025-03-25T23:00:42.104ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
9872025-03-25T23:00:42.104ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
9882025-03-25T23:00:42.105ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
9892025-03-25T23:00:42.105ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 108.451595ms iteration = 23 status = 0
9902025-03-25T23:00:42.105ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9912025-03-25T23:00:42.106ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 39030d55-287b-484a-98ee-50f0b968e055 dns_zone_name = control-plane.oxide.internal found_so_far = 52 version = 1
9922025-03-25T23:00:42.106ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
9932025-03-25T23:00:42.106ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2025-03-25 22:49:42.354168 UTC dns_group = internal generation = 1
9942025-03-25T23:00:42.106ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 110.362867ms iteration = 12 status = {"generation":1}
9952025-03-25T23:00:42.106ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
9962025-03-25T23:00:42.106ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
9972025-03-25T23:00:42.107ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
9982025-03-25T23:00:42.107ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
9992025-03-25T23:00:42.108ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
10002025-03-25T23:00:42.108ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10012025-03-25T23:00:42.109ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
10022025-03-25T23:00:42.109ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:140 found = 0 not_ready_to_be_deleted = 0
10032025-03-25T23:00:42.109ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 112.315661ms iteration = 12 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
10042025-03-25T23:00:42.110ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
10052025-03-25T23:00:42.110ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
10062025-03-25T23:00:42.110ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
10072025-03-25T23:00:42.111ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
10082025-03-25T23:00:42.111ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
10092025-03-25T23:00:42.112ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 115.845882ms iteration = 23 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
10102025-03-25T23:00:42.113ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
10112025-03-25T23:00:42.113ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 6f07f433-517a-46c5-a5fb-dffdd0b3ba20 dns_zone_name = oxide.test found_so_far = 0 version = 2
10122025-03-25T23:00:42.113ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
10132025-03-25T23:00:42.113ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
10142025-03-25T23:00:42.114ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10152025-03-25T23:00:42.115ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
10162025-03-25T23:00:42.116ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1085 zone_address = fd00:1122:3344:101::2
10172025-03-25T23:00:42.116ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
10182025-03-25T23:00:42.116ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 120.072867ms iteration = 23 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
10192025-03-25T23:00:42.117ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "e3c3dee2-f3e0-421e-88c5-02781eb860df", "content-length": "26", "date": "Tue, 25 Mar 2025 23:00:42 GMT"} })
10202025-03-25T23:00:42.117ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1088 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
10212025-03-25T23:00:42.118ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
10222025-03-25T23:00:42.118ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
10232025-03-25T23:00:42.118ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
10242025-03-25T23:00:42.118ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1122 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
10252025-03-25T23:00:42.120ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 6f07f433-517a-46c5-a5fb-dffdd0b3ba20 dns_zone_name = oxide.test found_so_far = 1 version = 2
10262025-03-25T23:00:42.120ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
10272025-03-25T23:00:42.120ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2025-03-25 22:49:43.689588 UTC dns_group = external generation = 2
10282025-03-25T23:00:42.120ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 124.551258ms iteration = 12 status = {"generation":2}
10292025-03-25T23:00:42.140ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
10302025-03-25T23:00:42.141ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
10312025-03-25T23:00:42.142ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
10322025-03-25T23:00:42.142ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
10332025-03-25T23:00:42.142ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 145.538241ms iteration = 23 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
10342025-03-25T23:00:42.150ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
10352025-03-25T23:00:42.150ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
10362025-03-25T23:00:42.153ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:360
10372025-03-25T23:00:42.153ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 156.5866ms iteration = 23 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
10382025-03-25T23:00:42.154ZINFOnexus (BackgroundTasks): found Silo/DNS/TLS config (changed) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2025-03-25T22:49:38.852077Z, time_modified: 2025-03-25T22:49:38.852077Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 526efdec-95ea-41d0-8852-ffb890a1c1a5, db_silo: Silo { identity: SiloIdentity { id: 526efdec-95ea-41d0-8852-ffb890a1c1a5, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-03-25T22:49:43.626129Z, time_modified: 2025-03-25T22:49:43.626129Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "8a79e1b9407ebe7db0e8d46e699071dda0ee97f80cecb334287dbf0849e0f965" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 526efdec-95ea-41d0-8852-ffb890a1c1a5, db_silo: Silo { identity: SiloIdentity { id: 526efdec-95ea-41d0-8852-ffb890a1c1a5, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-03-25T22:49:43.626129Z, time_modified: 2025-03-25T22:49:43.626129Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "8a79e1b9407ebe7db0e8d46e699071dda0ee97f80cecb334287dbf0849e0f965" }] }) } file = nexus/src/app/background/tasks/external_endpoints.rs:94
10392025-03-25T23:00:42.154ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 159.199178ms iteration = 12 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"526efdec-95ea-41d0-8852-ffb890a1c1a5","tls_certs":["8a79e1b9407ebe7db0e8d46e699071dda0ee97f80cecb334287dbf0849e0f965"]}},"default_endpoint":{"silo_id":"526efdec-95ea-41d0-8852-ffb890a1c1a5","tls_certs":["8a79e1b9407ebe7db0e8d46e699071dda0ee97f80cecb334287dbf0849e0f965"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
10402025-03-25T23:00:42.220ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10412025-03-25T23:00:42.224ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
10422025-03-25T23:00:42.224ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
10432025-03-25T23:00:42.227ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
10442025-03-25T23:00:42.228ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10452025-03-25T23:00:42.231ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
10462025-03-25T23:00:42.231ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
10472025-03-25T23:00:42.232ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
10482025-03-25T23:00:42.239ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1085 zone_address = fd00:1122:3344:101::2
10492025-03-25T23:00:42.239ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
10502025-03-25T23:00:42.240ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "a3f22b8b-d796-4aa9-ae67-6ca73132f401", "content-length": "26", "date": "Tue, 25 Mar 2025 23:00:42 GMT"} })
10512025-03-25T23:00:42.240ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1088 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
10522025-03-25T23:00:42.243ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1122 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
10532025-03-25T23:00:42.248ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 251.834067ms iteration = 26 status = {"physical_disks_added":0}
10542025-03-25T23:00:42.261ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1085 rack_id = 3ef3cc6b-3ffd-47a3-99ef-c57b25776619 zone_address = fd00:1122:3344:101::2
10552025-03-25T23:00:42.263ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 3ef3cc6b-3ffd-47a3-99ef-c57b25776619 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
10562025-03-25T23:00:42.264ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 3ef3cc6b-3ffd-47a3-99ef-c57b25776619 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "f59f5a7b-0248-4a39-924c-d9e321e9004d", "content-length": "26", "date": "Tue, 25 Mar 2025 23:00:42 GMT"} })
10572025-03-25T23:00:42.264ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1088 rack_id = 3ef3cc6b-3ffd-47a3-99ef-c57b25776619 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
10582025-03-25T23:00:42.265ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1122 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 3ef3cc6b-3ffd-47a3-99ef-c57b25776619
10592025-03-25T23:00:42.272ZWARNnexus (BackgroundTasks): instance reincarnation completed with errors background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:103 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0 query_errors = 0 restart_errors = 0
10602025-03-25T23:00:42.272ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 275.49804ms iteration = 12 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
10612025-03-25T23:00:42.307ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
10622025-03-25T23:00:42.372ZINFOnexus (BackgroundTasks): VPC route manager sled 9af3e790-030d-4974-9b9e-c35f829f8643 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
10632025-03-25T23:00:42.372ZDEBGnexus (BackgroundTasks): client request SledAgent = 9af3e790-030d-4974-9b9e-c35f829f8643 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
10642025-03-25T23:00:42.372ZDEBGnexus (BackgroundTasks): client response SledAgent = 9af3e790-030d-4974-9b9e-c35f829f8643 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "83a1fad8-b719-462f-b47c-50f1eddd4651", "content-length": "372", "date": "Tue, 25 Mar 2025 23:00:42 GMT"} })
10652025-03-25T23:00:42.372ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
10662025-03-25T23:00:42.373ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
10672025-03-25T23:00:42.380ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 9a38eebf-2600-4a66-90fd-87129be95bb5 roles = RoleSet { roles: {} } uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
10682025-03-25T23:00:42.381ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 9a38eebf-2600-4a66-90fd-87129be95bb5 resource = Database result = Ok(()) uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
10692025-03-25T23:00:42.382ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1964 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 9a38eebf-2600-4a66-90fd-87129be95bb5 response_code = 200 uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
10702025-03-25T23:00:42.389ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 393.135371ms iteration = 23 status = {}
10712025-03-25T23:00:42.399ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
10722025-03-25T23:00:42.406ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
10732025-03-25T23:00:42.406ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "9367c9e0-c185-4c0e-948c-23decd54d678", "content-length": "2", "date": "Tue, 25 Mar 2025 23:00:41 GMT"} })
10742025-03-25T23:00:42.412ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
10752025-03-25T23:00:42.412ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
10762025-03-25T23:00:42.416ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 420.951889ms iteration = 23 status = null
10772025-03-25T23:00:42.423ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = buskin
10782025-03-25T23:00:42.423ZDEBGnexus (BackgroundTasks): client request SledAgent = 9af3e790-030d-4974-9b9e-c35f829f8643 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
10792025-03-25T23:00:42.423ZDEBGnexus (BackgroundTasks): client response SledAgent = 9af3e790-030d-4974-9b9e-c35f829f8643 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "8523d316-a334-40a2-bd50-fff163697da8", "date": "Tue, 25 Mar 2025 23:00:42 GMT"} })
10802025-03-25T23:00:42.423ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
10812025-03-25T23:00:42.424ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
10822025-03-25T23:00:42.427ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
10832025-03-25T23:00:42.427ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
10842025-03-25T23:00:42.440ZDEBGnexus (BackgroundTasks): client request SledAgent = 9af3e790-030d-4974-9b9e-c35f829f8643 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
10852025-03-25T23:00:42.440ZDEBGnexus (BackgroundTasks): client response SledAgent = 9af3e790-030d-4974-9b9e-c35f829f8643 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "6c5aa78c-258b-4edd-a881-03635b82b9b3", "content-length": "691", "date": "Tue, 25 Mar 2025 23:00:42 GMT"} })
10862025-03-25T23:00:42.440ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = buskin
10872025-03-25T23:00:42.440ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = buskin
10882025-03-25T23:00:42.440ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = buskin
10892025-03-25T23:00:42.442ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 445.750308ms iteration = 24 status = {}
10902025-03-25T23:00:42.444ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
10912025-03-25T23:00:42.445ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
10922025-03-25T23:00:42.449ZINFOnexus (BackgroundTasks): VPC route manager sled 9af3e790-030d-4974-9b9e-c35f829f8643 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
10932025-03-25T23:00:42.449ZDEBGnexus (BackgroundTasks): client request SledAgent = 9af3e790-030d-4974-9b9e-c35f829f8643 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
10942025-03-25T23:00:42.449ZDEBGnexus (BackgroundTasks): client response SledAgent = 9af3e790-030d-4974-9b9e-c35f829f8643 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "1c02dedd-e4b9-4c32-93e1-17a7dd70d18e", "date": "Tue, 25 Mar 2025 23:00:42 GMT"} })
10952025-03-25T23:00:42.451ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 455.006296ms iteration = 24 status = {}
10962025-03-25T23:00:42.475ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
10972025-03-25T23:00:42.475ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "2431c002-92d5-40e2-83a2-b6a68200acae", "date": "Tue, 25 Mar 2025 23:00:42 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
10982025-03-25T23:00:42.475ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
10992025-03-25T23:00:42.476ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "c61c7baf-26a2-4bcc-b57e-a54ecdbf4172", "date": "Tue, 25 Mar 2025 23:00:42 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
11002025-03-25T23:00:42.476ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
11012025-03-25T23:00:42.478ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
11022025-03-25T23:00:42.478ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 482.589438ms iteration = 14 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
11032025-03-25T23:00:42.496ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
11042025-03-25T23:00:42.496ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "b955d815-939f-4592-9ec5-5f5caa4bb6e3", "date": "Tue, 25 Mar 2025 23:00:42 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
11052025-03-25T23:00:42.496ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
11062025-03-25T23:00:42.498ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "eacca018-7e6e-460d-aa8c-15766c7cc9fc", "date": "Tue, 25 Mar 2025 23:00:42 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
11072025-03-25T23:00:42.498ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
11082025-03-25T23:00:42.500ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "1175d4ca-8e3a-4339-a221-013ffd8d97b5", "date": "Tue, 25 Mar 2025 23:00:42 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
11092025-03-25T23:00:42.500ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
11102025-03-25T23:00:42.501ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
11112025-03-25T23:00:42.501ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 506.127799ms iteration = 14 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
11122025-03-25T23:00:42.534ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
11132025-03-25T23:00:42.534ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
11142025-03-25T23:00:42.536ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: 25f1877d-a731-48fe-8967-16ef2fb00f3b, rack_id: 3ef3cc6b-3ffd-47a3-99ef-c57b25776619, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(dcb3f9bf-5ea7-47e3-b699-6c3a8c7459c5) } rack_id = 3ef3cc6b-3ffd-47a3-99ef-c57b25776619 switch_location = Switch0
11152025-03-25T23:00:42.536ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
11162025-03-25T23:00:42.537ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
11172025-03-25T23:00:42.546ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 3ef3cc6b-3ffd-47a3-99ef-c57b25776619 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
11182025-03-25T23:00:42.546ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 3ef3cc6b-3ffd-47a3-99ef-c57b25776619 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "ce271e5f-9866-41d4-b647-a3879c8b66e8", "content-length": "134", "date": "Tue, 25 Mar 2025 23:00:41 GMT"} })
11192025-03-25T23:00:42.546ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1741 rack_id = 3ef3cc6b-3ffd-47a3-99ef-c57b25776619 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
11202025-03-25T23:00:42.546ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1768 rack_id = 3ef3cc6b-3ffd-47a3-99ef-c57b25776619 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = dcb3f9bf-5ea7-47e3-b699-6c3a8c7459c5
11212025-03-25T23:00:42.547ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:378 rack_id = 3ef3cc6b-3ffd-47a3-99ef-c57b25776619 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
11222025-03-25T23:00:42.547ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:382 rack_id = 3ef3cc6b-3ffd-47a3-99ef-c57b25776619 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
11232025-03-25T23:00:42.547ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:391 rack_id = 3ef3cc6b-3ffd-47a3-99ef-c57b25776619 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
11242025-03-25T23:00:42.547ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:397 rack_id = 3ef3cc6b-3ffd-47a3-99ef-c57b25776619 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
11252025-03-25T23:00:42.547ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = 3ef3cc6b-3ffd-47a3-99ef-c57b25776619 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
11262025-03-25T23:00:42.547ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1915 rack_id = 3ef3cc6b-3ffd-47a3-99ef-c57b25776619 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
11272025-03-25T23:00:42.547ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:408 rack_id = 3ef3cc6b-3ffd-47a3-99ef-c57b25776619 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
11282025-03-25T23:00:42.547ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1951 rack_id = 3ef3cc6b-3ffd-47a3-99ef-c57b25776619 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
11292025-03-25T23:00:42.547ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:417 rack_id = 3ef3cc6b-3ffd-47a3-99ef-c57b25776619
11302025-03-25T23:00:42.547ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
11312025-03-25T23:00:42.547ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
11322025-03-25T23:00:42.549ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 3ef3cc6b-3ffd-47a3-99ef-c57b25776619 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
11332025-03-25T23:00:42.549ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 3ef3cc6b-3ffd-47a3-99ef-c57b25776619 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "fc0533f6-ccc0-448d-a83b-9b1ed9c16705", "content-length": "2", "date": "Tue, 25 Mar 2025 23:00:41 GMT"} })
11342025-03-25T23:00:42.549ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 3ef3cc6b-3ffd-47a3-99ef-c57b25776619 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
11352025-03-25T23:00:42.549ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 3ef3cc6b-3ffd-47a3-99ef-c57b25776619 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "0c62a2bf-aed3-4728-aac6-c858b87256e1", "content-length": "52", "date": "Tue, 25 Mar 2025 23:00:41 GMT"} })
11362025-03-25T23:00:42.549ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.1.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:466 rack_id = 3ef3cc6b-3ffd-47a3-99ef-c57b25776619 switch_location = Switch0
11372025-03-25T23:00:42.549ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 3ef3cc6b-3ffd-47a3-99ef-c57b25776619 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
11382025-03-25T23:00:42.659ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 3ef3cc6b-3ffd-47a3-99ef-c57b25776619 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "cafe573d-0784-4c20-afa9-78993a4b5c30", "date": "Tue, 25 Mar 2025 23:00:42 GMT"} })
11392025-03-25T23:00:42.659ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:769 rack_id = 3ef3cc6b-3ffd-47a3-99ef-c57b25776619 switch = Switch0
11402025-03-25T23:00:42.659ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 3ef3cc6b-3ffd-47a3-99ef-c57b25776619 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
11412025-03-25T23:00:42.663ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 3ef3cc6b-3ffd-47a3-99ef-c57b25776619 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "d5c5f4da-c5d2-4cfc-8bb6-a64ae84fb947", "content-length": "682", "date": "Tue, 25 Mar 2025 23:00:42 GMT"} })
11422025-03-25T23:00:42.663ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
11432025-03-25T23:00:42.664ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
11442025-03-25T23:00:42.667ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
11452025-03-25T23:00:42.668ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
11462025-03-25T23:00:42.671ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
11472025-03-25T23:00:42.671ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
11482025-03-25T23:00:42.673ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
11492025-03-25T23:00:42.673ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
11502025-03-25T23:00:42.680ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 683.925817ms iteration = 23 status = {}
11512025-03-25T23:00:42.768ZDEBGnexus (producer-registration-task): registering / renewing oximeter producer lease with Nexus
11522025-03-25T23:00:42.768ZDEBGnexus (producer-registration-task): using nexus address for registration addr = [fd00:1122:3344:101::a]:12221
11532025-03-25T23:00:42.800ZDEBGnexus (producer-registration-task): client request body = Some(Body) method = POST uri = http://[fd00:1122:3344:101::a]:12221/metrics/producers
11542025-03-25T23:00:42.800ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:1025 local_addr = [fd00:1122:3344:101::a]:12221 remote_addr = [fd00:1122:3344:101::a]:60127
11552025-03-25T23:00:42.800ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = POST remote_addr = [fd00:1122:3344:101::a]:60127 req_id = cc7f8a3d-49c1-4360-8d6a-1e6bb19e8f14 roles = RoleSet { roles: {} } uri = /metrics/producers
11562025-03-25T23:00:42.801ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = POST remote_addr = [fd00:1122:3344:101::a]:60127 req_id = cc7f8a3d-49c1-4360-8d6a-1e6bb19e8f14 resource = Database result = Ok(()) uri = /metrics/producers
11572025-03-25T23:00:42.839ZINFOnexus: assigned collector to new producer collector_id = aa92c5e2-2c4d-4351-b468-f1bc420a118d file = nexus/src/app/oximeter.rs:76 producer_id = 8f96aade-61ab-4710-a1b1-0efefa6704ee
11582025-03-25T23:00:42.839ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 38416 local_addr = [fd00:1122:3344:101::a]:12221 method = POST remote_addr = [fd00:1122:3344:101::a]:60127 req_id = cc7f8a3d-49c1-4360-8d6a-1e6bb19e8f14 response_code = 201 uri = /metrics/producers
11592025-03-25T23:00:42.839ZDEBGnexus (producer-registration-task): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::a]:12221/metrics/producers", status: 201, headers: {"content-type": "application/json", "x-request-id": "cc7f8a3d-49c1-4360-8d6a-1e6bb19e8f14", "content-length": "41", "date": "Tue, 25 Mar 2025 23:00:42 GMT"} })
11602025-03-25T23:00:42.841ZDEBGnexus (producer-registration-task): registered with nexus successfully
11612025-03-25T23:00:42.841ZDEBGnexus (producer-registration-task): pausing until time to renew lease lease_duration = 600s wait_period = 150s
11622025-03-25T23:00:43.407ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 47688cfe-fc23-4d37-8c64-4c3982603f46 roles = RoleSet { roles: {} } uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
11632025-03-25T23:00:43.407ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 47688cfe-fc23-4d37-8c64-4c3982603f46 resource = Database result = Ok(()) uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
11642025-03-25T23:00:43.409ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2246 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 47688cfe-fc23-4d37-8c64-4c3982603f46 response_code = 200 uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
11652025-03-25T23:00:43.856ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 644 local_addr = [fd00:1122:3344:101::a]:62174 method = GET remote_addr = [fd00:1122:3344:101::d]:63115 req_id = 8b9b51f6-661e-4f5e-ac81-d45ea0d91dfe response_code = 200 uri = /8f96aade-61ab-4710-a1b1-0efefa6704ee
11662025-03-25T23:00:44.429ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 49ca6f69-6f5e-4f27-bf01-d00b30a82630 roles = RoleSet { roles: {} } uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
11672025-03-25T23:00:44.429ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 49ca6f69-6f5e-4f27-bf01-d00b30a82630 resource = Database result = Ok(()) uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
11682025-03-25T23:00:44.431ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2167 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 49ca6f69-6f5e-4f27-bf01-d00b30a82630 response_code = 200 uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
11692025-03-25T23:00:45.450ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 031b9f54-aaeb-474d-aa96-ea4d35673ce0 roles = RoleSet { roles: {} } uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
11702025-03-25T23:00:45.451ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 031b9f54-aaeb-474d-aa96-ea4d35673ce0 resource = Database result = Ok(()) uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
11712025-03-25T23:00:45.452ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1966 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 031b9f54-aaeb-474d-aa96-ea4d35673ce0 response_code = 200 uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
11722025-03-25T23:00:46.472ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = d0bc2830-3354-44b4-8dcc-0bdd097c5285 roles = RoleSet { roles: {} } uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
11732025-03-25T23:00:46.473ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = d0bc2830-3354-44b4-8dcc-0bdd097c5285 resource = Database result = Ok(()) uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
11742025-03-25T23:00:46.475ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2394 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = d0bc2830-3354-44b4-8dcc-0bdd097c5285 response_code = 200 uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
11752025-03-25T23:00:47.495ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 21e1c305-b430-4435-bdd8-d2bb9a88a678 roles = RoleSet { roles: {} } uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
11762025-03-25T23:00:47.495ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 21e1c305-b430-4435-bdd8-d2bb9a88a678 resource = Database result = Ok(()) uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
11772025-03-25T23:00:47.497ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2378 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 21e1c305-b430-4435-bdd8-d2bb9a88a678 response_code = 200 uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
11782025-03-25T23:00:48.518ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 4e1694cf-068b-4d6b-a7f5-dceca17159b2 roles = RoleSet { roles: {} } uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
11792025-03-25T23:00:48.518ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 4e1694cf-068b-4d6b-a7f5-dceca17159b2 resource = Database result = Ok(()) uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
11802025-03-25T23:00:48.520ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2273 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 4e1694cf-068b-4d6b-a7f5-dceca17159b2 response_code = 200 uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
11812025-03-25T23:00:49.540ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = fa858101-684d-406a-9f1f-1bd954e3dafd roles = RoleSet { roles: {} } uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
11822025-03-25T23:00:49.541ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = fa858101-684d-406a-9f1f-1bd954e3dafd resource = Database result = Ok(()) uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
11832025-03-25T23:00:49.543ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2522 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = fa858101-684d-406a-9f1f-1bd954e3dafd response_code = 200 uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
11842025-03-25T23:00:50.563ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 8ff0132d-1dc7-4f27-a689-0e28d96022dd roles = RoleSet { roles: {} } uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
11852025-03-25T23:00:50.564ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 8ff0132d-1dc7-4f27-a689-0e28d96022dd resource = Database result = Ok(()) uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
11862025-03-25T23:00:50.566ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2399 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 8ff0132d-1dc7-4f27-a689-0e28d96022dd response_code = 200 uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
11872025-03-25T23:00:51.587ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = f4594a50-089b-4cf7-97ac-92b55a2c44aa roles = RoleSet { roles: {} } uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
11882025-03-25T23:00:51.587ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = f4594a50-089b-4cf7-97ac-92b55a2c44aa resource = Database result = Ok(()) uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
11892025-03-25T23:00:51.589ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2791 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = f4594a50-089b-4cf7-97ac-92b55a2c44aa response_code = 200 uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
11902025-03-25T23:00:51.996ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 68 reason = Timeout
11912025-03-25T23:00:51.996ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
11922025-03-25T23:00:51.996ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
11932025-03-25T23:00:51.996ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 68 reason = Timeout
11942025-03-25T23:00:51.996ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
11952025-03-25T23:00:51.997ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
11962025-03-25T23:00:51.998ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11972025-03-25T23:00:52.000ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
11982025-03-25T23:00:52.000ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
11992025-03-25T23:00:52.001ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
12002025-03-25T23:00:52.003ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
12012025-03-25T23:00:52.003ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
12022025-03-25T23:00:52.005ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12032025-03-25T23:00:52.007ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 5d9d6f43-fd71-4256-b06d-1ed4c87e191f, lookup_type: ById(5d9d6f43-fd71-4256-b06d-1ed4c87e191f) } result = Ok(())
12042025-03-25T23:00:52.007ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
12052025-03-25T23:00:52.008ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
12062025-03-25T23:00:52.018ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 5d9d6f43-fd71-4256-b06d-1ed4c87e191f original_time_created = 2025-03-25 22:49:35.383803 UTC target_id = 5d9d6f43-fd71-4256-b06d-1ed4c87e191f time_created = 2025-03-25 22:49:35.383803 UTC
12072025-03-25T23:00:52.018ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 22.886886ms iteration = 68 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"5d9d6f43-fd71-4256-b06d-1ed4c87e191f","time_created":"2025-03-25T22:49:35.383803Z"}
12082025-03-25T23:00:52.027ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
12092025-03-25T23:00:52.027ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
12102025-03-25T23:00:52.029ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 32.357ms iteration = 68 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
12112025-03-25T23:00:52.610ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = bd14c7dd-9caf-4bce-8c8e-4a29357551da roles = RoleSet { roles: {} } uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
12122025-03-25T23:00:52.610ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = bd14c7dd-9caf-4bce-8c8e-4a29357551da resource = Database result = Ok(()) uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
12132025-03-25T23:00:52.612ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2409 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = bd14c7dd-9caf-4bce-8c8e-4a29357551da response_code = 200 uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
12142025-03-25T23:00:53.632ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 786353b3-7e5e-4b5b-b683-88fdeaee2db6 roles = RoleSet { roles: {} } uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
12152025-03-25T23:00:53.632ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 786353b3-7e5e-4b5b-b683-88fdeaee2db6 resource = Database result = Ok(()) uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
12162025-03-25T23:00:53.634ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2304 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 786353b3-7e5e-4b5b-b683-88fdeaee2db6 response_code = 200 uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
12172025-03-25T23:00:53.856ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 889 local_addr = [fd00:1122:3344:101::a]:62174 method = GET remote_addr = [fd00:1122:3344:101::d]:63115 req_id = b55e6790-3fb5-4fa7-95d3-babc1ce86004 response_code = 200 uri = /8f96aade-61ab-4710-a1b1-0efefa6704ee
12182025-03-25T23:00:54.655ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 15b9ea3b-9239-4395-8f15-8cbf0f30abd3 roles = RoleSet { roles: {} } uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
12192025-03-25T23:00:54.655ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 15b9ea3b-9239-4395-8f15-8cbf0f30abd3 resource = Database result = Ok(()) uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
12202025-03-25T23:00:54.657ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1813 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 15b9ea3b-9239-4395-8f15-8cbf0f30abd3 response_code = 200 uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
12212025-03-25T23:00:55.677ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = f415d19e-3127-4c13-a64f-129e86690db4 roles = RoleSet { roles: {} } uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
12222025-03-25T23:00:55.678ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = f415d19e-3127-4c13-a64f-129e86690db4 resource = Database result = Ok(()) uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
12232025-03-25T23:00:55.680ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2358 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = f415d19e-3127-4c13-a64f-129e86690db4 response_code = 200 uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
12242025-03-25T23:00:56.700ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 74fd1ce4-0e9e-46f1-852c-417753248876 roles = RoleSet { roles: {} } uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
12252025-03-25T23:00:56.700ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 74fd1ce4-0e9e-46f1-852c-417753248876 resource = Database result = Ok(()) uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
12262025-03-25T23:00:56.702ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2382 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 74fd1ce4-0e9e-46f1-852c-417753248876 response_code = 200 uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
12272025-03-25T23:00:57.722ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 3ecf4e41-c8f3-407f-8166-8b3bdf988576 roles = RoleSet { roles: {} } uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
12282025-03-25T23:00:57.723ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 3ecf4e41-c8f3-407f-8166-8b3bdf988576 resource = Database result = Ok(()) uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
12292025-03-25T23:00:57.725ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2474 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 3ecf4e41-c8f3-407f-8166-8b3bdf988576 response_code = 200 uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
12302025-03-25T23:00:58.745ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 4fb28dd3-61ba-47e8-98c3-1e57dca672e0 roles = RoleSet { roles: {} } uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
12312025-03-25T23:00:58.745ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 4fb28dd3-61ba-47e8-98c3-1e57dca672e0 resource = Database result = Ok(()) uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
12322025-03-25T23:00:58.747ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2202 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 4fb28dd3-61ba-47e8-98c3-1e57dca672e0 response_code = 200 uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
12332025-03-25T23:00:59.769ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 2bda3e0a-5e11-4694-ab05-6bba231107fb roles = RoleSet { roles: {} } uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
12342025-03-25T23:00:59.769ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 2bda3e0a-5e11-4694-ab05-6bba231107fb resource = Database result = Ok(()) uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
12352025-03-25T23:00:59.771ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2107 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 2bda3e0a-5e11-4694-ab05-6bba231107fb response_code = 200 uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
12362025-03-25T23:01:00.792ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 99d8c99e-4255-474e-969f-1ab45a2d9041 roles = RoleSet { roles: {} } uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
12372025-03-25T23:01:00.792ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 99d8c99e-4255-474e-969f-1ab45a2d9041 resource = Database result = Ok(()) uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
12382025-03-25T23:01:00.794ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2459 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 99d8c99e-4255-474e-969f-1ab45a2d9041 response_code = 200 uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
12392025-03-25T23:01:01.816ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 55a2f5d0-d1ee-48f9-a43d-810959b15608 roles = RoleSet { roles: {} } uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
12402025-03-25T23:01:01.817ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 55a2f5d0-d1ee-48f9-a43d-810959b15608 resource = Database result = Ok(()) uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
12412025-03-25T23:01:01.819ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2620 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 55a2f5d0-d1ee-48f9-a43d-810959b15608 response_code = 200 uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
12422025-03-25T23:01:01.996ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 69 reason = Timeout
12432025-03-25T23:01:01.996ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
12442025-03-25T23:01:01.996ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
12452025-03-25T23:01:01.997ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 69 reason = Timeout
12462025-03-25T23:01:01.997ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
12472025-03-25T23:01:01.997ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
12482025-03-25T23:01:01.999ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12492025-03-25T23:01:01.999ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
12502025-03-25T23:01:01.999ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
12512025-03-25T23:01:02.001ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 4.173384ms iteration = 69 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
12522025-03-25T23:01:02.001ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
12532025-03-25T23:01:02.001ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
12542025-03-25T23:01:02.002ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
12552025-03-25T23:01:02.003ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
12562025-03-25T23:01:02.004ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
12572025-03-25T23:01:02.005ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12582025-03-25T23:01:02.008ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 5d9d6f43-fd71-4256-b06d-1ed4c87e191f, lookup_type: ById(5d9d6f43-fd71-4256-b06d-1ed4c87e191f) } result = Ok(())
12592025-03-25T23:01:02.008ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
12602025-03-25T23:01:02.008ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
12612025-03-25T23:01:02.015ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 5d9d6f43-fd71-4256-b06d-1ed4c87e191f original_time_created = 2025-03-25 22:49:35.383803 UTC target_id = 5d9d6f43-fd71-4256-b06d-1ed4c87e191f time_created = 2025-03-25 22:49:35.383803 UTC
12622025-03-25T23:01:02.015ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 18.695802ms iteration = 69 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"5d9d6f43-fd71-4256-b06d-1ed4c87e191f","time_created":"2025-03-25T22:49:35.383803Z"}
12632025-03-25T23:01:02.839ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = f4a3ac56-f597-4bac-ba4e-c64b5a27bfaf roles = RoleSet { roles: {} } uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
12642025-03-25T23:01:02.839ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = f4a3ac56-f597-4bac-ba4e-c64b5a27bfaf resource = Database result = Ok(()) uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
12652025-03-25T23:01:02.841ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2127 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = f4a3ac56-f597-4bac-ba4e-c64b5a27bfaf response_code = 200 uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
12662025-03-25T23:01:03.856ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 823 local_addr = [fd00:1122:3344:101::a]:62174 method = GET remote_addr = [fd00:1122:3344:101::d]:63115 req_id = 6a2fd3a2-50cf-4f40-ab74-61cc78af1ceb response_code = 200 uri = /8f96aade-61ab-4710-a1b1-0efefa6704ee
12672025-03-25T23:01:03.860ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = ff3f282b-a232-4829-98c7-5ea56756ad21 roles = RoleSet { roles: {} } uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
12682025-03-25T23:01:03.861ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = ff3f282b-a232-4829-98c7-5ea56756ad21 resource = Database result = Ok(()) uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
12692025-03-25T23:01:03.862ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1799 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = ff3f282b-a232-4829-98c7-5ea56756ad21 response_code = 200 uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
12702025-03-25T23:01:04.882ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = d9bc0253-776a-4e67-87be-89944d3b7ba6 roles = RoleSet { roles: {} } uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
12712025-03-25T23:01:04.882ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = d9bc0253-776a-4e67-87be-89944d3b7ba6 resource = Database result = Ok(()) uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
12722025-03-25T23:01:04.885ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2718 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = d9bc0253-776a-4e67-87be-89944d3b7ba6 response_code = 200 uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
12732025-03-25T23:01:05.905ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = df49d9af-0b63-4d0d-b88c-fbb1c2e218c3 roles = RoleSet { roles: {} } uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
12742025-03-25T23:01:05.905ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = df49d9af-0b63-4d0d-b88c-fbb1c2e218c3 resource = Database result = Ok(()) uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
12752025-03-25T23:01:05.907ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2111 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = df49d9af-0b63-4d0d-b88c-fbb1c2e218c3 response_code = 200 uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
12762025-03-25T23:01:06.927ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = f6e9b141-060b-4da6-a9d1-a11fdc605516 roles = RoleSet { roles: {} } uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
12772025-03-25T23:01:06.927ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = f6e9b141-060b-4da6-a9d1-a11fdc605516 resource = Database result = Ok(()) uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
12782025-03-25T23:01:06.929ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2594 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = f6e9b141-060b-4da6-a9d1-a11fdc605516 response_code = 200 uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
12792025-03-25T23:01:07.949ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 363fac10-dc13-48f3-b846-ce2270aab344 roles = RoleSet { roles: {} } uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
12802025-03-25T23:01:07.950ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 363fac10-dc13-48f3-b846-ce2270aab344 resource = Database result = Ok(()) uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
12812025-03-25T23:01:07.952ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2434 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 363fac10-dc13-48f3-b846-ce2270aab344 response_code = 200 uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
12822025-03-25T23:01:08.841ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:1025 local_addr = [fd00:1122:3344:101::a]:12221 remote_addr = [fd00:1122:3344:101::d]:40812
12832025-03-25T23:01:08.841ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:40812 req_id = 48cb5130-d028-4bbc-9339-bb5727d3351f roles = RoleSet { roles: {} } uri = /metrics/collectors/aa92c5e2-2c4d-4351-b468-f1bc420a118d/producers?sort_by=id_ascending
12842025-03-25T23:01:08.841ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:40812 req_id = 48cb5130-d028-4bbc-9339-bb5727d3351f resource = Database result = Ok(()) uri = /metrics/collectors/aa92c5e2-2c4d-4351-b468-f1bc420a118d/producers?sort_by=id_ascending
12852025-03-25T23:01:08.844ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 3356 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:40812 req_id = 48cb5130-d028-4bbc-9339-bb5727d3351f response_code = 200 uri = /metrics/collectors/aa92c5e2-2c4d-4351-b468-f1bc420a118d/producers?sort_by=id_ascending
12862025-03-25T23:01:08.844ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:40812 req_id = 59a612a3-3915-4f2d-9da9-7a0f0120fdc3 roles = RoleSet { roles: {} } uri = /metrics/collectors/aa92c5e2-2c4d-4351-b468-f1bc420a118d/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMzQ4MTE5Yy05YTdjLTRiNWQtOGNkOC1mZWRkZjQ3NjU5ZDUifX0%3D
12872025-03-25T23:01:08.845ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:40812 req_id = 59a612a3-3915-4f2d-9da9-7a0f0120fdc3 resource = Database result = Ok(()) uri = /metrics/collectors/aa92c5e2-2c4d-4351-b468-f1bc420a118d/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMzQ4MTE5Yy05YTdjLTRiNWQtOGNkOC1mZWRkZjQ3NjU5ZDUifX0%3D
12882025-03-25T23:01:08.847ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2431 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:40812 req_id = 59a612a3-3915-4f2d-9da9-7a0f0120fdc3 response_code = 200 uri = /metrics/collectors/aa92c5e2-2c4d-4351-b468-f1bc420a118d/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMzQ4MTE5Yy05YTdjLTRiNWQtOGNkOC1mZWRkZjQ3NjU5ZDUifX0%3D
12892025-03-25T23:01:08.972ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 42a0d03d-05e3-466c-afed-b72506974428 roles = RoleSet { roles: {} } uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
12902025-03-25T23:01:08.973ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 42a0d03d-05e3-466c-afed-b72506974428 resource = Database result = Ok(()) uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
12912025-03-25T23:01:08.975ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2329 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 42a0d03d-05e3-466c-afed-b72506974428 response_code = 200 uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
12922025-03-25T23:01:09.995ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 71583de9-7c6f-49f5-8d8c-4ed5286a94cd roles = RoleSet { roles: {} } uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
12932025-03-25T23:01:09.995ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 71583de9-7c6f-49f5-8d8c-4ed5286a94cd resource = Database result = Ok(()) uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
12942025-03-25T23:01:09.997ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2237 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 71583de9-7c6f-49f5-8d8c-4ed5286a94cd response_code = 200 uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
12952025-03-25T23:01:11.018ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 488e1801-6092-4be8-86bf-4b692d573027 roles = RoleSet { roles: {} } uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
12962025-03-25T23:01:11.019ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 488e1801-6092-4be8-86bf-4b692d573027 resource = Database result = Ok(()) uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
12972025-03-25T23:01:11.020ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2331 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 488e1801-6092-4be8-86bf-4b692d573027 response_code = 200 uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
12982025-03-25T23:01:11.996ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 70 reason = Timeout
12992025-03-25T23:01:11.996ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 24 reason = Timeout
13002025-03-25T23:01:11.996ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
13012025-03-25T23:01:11.996ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 24 reason = Timeout
13022025-03-25T23:01:11.996ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
13032025-03-25T23:01:11.996ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
13042025-03-25T23:01:11.996ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 24 reason = Timeout
13052025-03-25T23:01:11.996ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
13062025-03-25T23:01:11.996ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
13072025-03-25T23:01:11.996ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
13082025-03-25T23:01:11.997ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 70 reason = Timeout
13092025-03-25T23:01:11.997ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 25 reason = Timeout
13102025-03-25T23:01:11.997ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
13112025-03-25T23:01:11.997ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
13122025-03-25T23:01:11.997ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 24 reason = Timeout
13132025-03-25T23:01:11.997ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 24 reason = Timeout
13142025-03-25T23:01:11.997ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
13152025-03-25T23:01:11.997ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 24 reason = Timeout
13162025-03-25T23:01:11.997ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
13172025-03-25T23:01:11.997ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
13182025-03-25T23:01:11.997ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 24 reason = Timeout
13192025-03-25T23:01:11.997ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
13202025-03-25T23:01:11.997ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 27 reason = Timeout
13212025-03-25T23:01:11.997ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
13222025-03-25T23:01:11.997ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 24 reason = Timeout
13232025-03-25T23:01:11.997ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
13242025-03-25T23:01:11.997ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 24 reason = Timeout
13252025-03-25T23:01:11.997ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
13262025-03-25T23:01:11.997ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
13272025-03-25T23:01:11.997ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 25 reason = Timeout
13282025-03-25T23:01:11.997ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
13292025-03-25T23:01:11.997ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 24 reason = Timeout
13302025-03-25T23:01:11.997ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
13312025-03-25T23:01:11.997ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
13322025-03-25T23:01:11.997ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 24 reason = Timeout
13332025-03-25T23:01:11.997ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
13342025-03-25T23:01:11.997ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 24 reason = Timeout
13352025-03-25T23:01:11.997ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 24 reason = Timeout
13362025-03-25T23:01:11.997ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
13372025-03-25T23:01:11.997ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
13382025-03-25T23:01:11.997ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 24 reason = Timeout
13392025-03-25T23:01:11.997ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
13402025-03-25T23:01:11.997ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
13412025-03-25T23:01:11.997ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
13422025-03-25T23:01:11.997ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
13432025-03-25T23:01:11.997ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
13442025-03-25T23:01:11.997ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
13452025-03-25T23:01:11.997ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
13462025-03-25T23:01:11.997ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
13472025-03-25T23:01:11.997ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
13482025-03-25T23:01:11.997ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
13492025-03-25T23:01:11.997ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
13502025-03-25T23:01:11.997ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
13512025-03-25T23:01:11.997ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
13522025-03-25T23:01:11.997ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
13532025-03-25T23:01:11.997ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
13542025-03-25T23:01:11.997ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
13552025-03-25T23:01:11.999ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13562025-03-25T23:01:11.999ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-9af3e790-030d-4974-9b9e-c35f829f8643.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-9af3e790-030d-4974-9b9e-c35f829f8643.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1845, tv_nsec: 445159925 } })
13572025-03-25T23:01:11.999ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1077
13582025-03-25T23:01:12.000ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-9af3e790-030d-4974-9b9e-c35f829f8643.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-9af3e790-030d-4974-9b9e-c35f829f8643.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1845, tv_nsec: 446159967 } })
13592025-03-25T23:01:12.000ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 4.338658ms iteration = 24 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
13602025-03-25T23:01:12.000ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1077
13612025-03-25T23:01:12.000ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13622025-03-25T23:01:12.001ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13632025-03-25T23:01:12.002ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
13642025-03-25T23:01:12.002ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
13652025-03-25T23:01:12.002ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
13662025-03-25T23:01:12.003ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
13672025-03-25T23:01:12.003ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
13682025-03-25T23:01:12.003ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
13692025-03-25T23:01:12.004ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
13702025-03-25T23:01:12.004ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
13712025-03-25T23:01:12.004ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
13722025-03-25T23:01:12.005ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
13732025-03-25T23:01:12.005ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
13742025-03-25T23:01:12.006ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
13752025-03-25T23:01:12.006ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
13762025-03-25T23:01:12.006ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
13772025-03-25T23:01:12.006ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
13782025-03-25T23:01:12.006ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13792025-03-25T23:01:12.007ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13802025-03-25T23:01:12.007ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13812025-03-25T23:01:12.008ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 11.277804ms iteration = 24 status = {"errors":[],"finish_invoked_ok":[]}
13822025-03-25T23:01:12.009ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
13832025-03-25T23:01:12.009ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
13842025-03-25T23:01:12.010ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
13852025-03-25T23:01:12.010ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
13862025-03-25T23:01:12.010ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
13872025-03-25T23:01:12.010ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
13882025-03-25T23:01:12.010ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
13892025-03-25T23:01:12.011ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
13902025-03-25T23:01:12.011ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
13912025-03-25T23:01:12.011ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
13922025-03-25T23:01:12.011ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
13932025-03-25T23:01:12.012ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13942025-03-25T23:01:12.014ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
13952025-03-25T23:01:12.014ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
13962025-03-25T23:01:12.015ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
13972025-03-25T23:01:12.015ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
13982025-03-25T23:01:12.015ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
13992025-03-25T23:01:12.015ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-9af3e790-030d-4974-9b9e-c35f829f8643.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-9af3e790-030d-4974-9b9e-c35f829f8643.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1845, tv_nsec: 461204496 } })
14002025-03-25T23:01:12.015ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
14012025-03-25T23:01:12.015ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
14022025-03-25T23:01:12.015ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1077 rack_id = 3ef3cc6b-3ffd-47a3-99ef-c57b25776619
14032025-03-25T23:01:12.016ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
14042025-03-25T23:01:12.016ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
14052025-03-25T23:01:12.018ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14062025-03-25T23:01:12.020ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
14072025-03-25T23:01:12.020ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
14082025-03-25T23:01:12.020ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
14092025-03-25T23:01:12.042ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 1c36ddd2-9d7c-4be9-8bb9-a7e612ecbc55 roles = RoleSet { roles: {} } uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
14102025-03-25T23:01:12.042ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 1c36ddd2-9d7c-4be9-8bb9-a7e612ecbc55 resource = Database result = Ok(()) uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
14112025-03-25T23:01:12.045ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
14122025-03-25T23:01:12.045ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
14132025-03-25T23:01:12.046ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 4449 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 1c36ddd2-9d7c-4be9-8bb9-a7e612ecbc55 response_code = 200 uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
14142025-03-25T23:01:12.048ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
14152025-03-25T23:01:12.048ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 50.938419ms iteration = 24 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
14162025-03-25T23:01:12.048ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
14172025-03-25T23:01:12.049ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
14182025-03-25T23:01:12.049ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
14192025-03-25T23:01:12.049ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 52.169956ms iteration = 24 status = {"errors":[],"garbage_collect_requested":[]}
14202025-03-25T23:01:12.050ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 52.908573ms iteration = 70 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
14212025-03-25T23:01:12.050ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
14222025-03-25T23:01:12.051ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
14232025-03-25T23:01:12.053ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
14242025-03-25T23:01:12.053ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
14252025-03-25T23:01:12.054ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 57.566537ms iteration = 24 status = {"errors":[],"requests_created_ok":[]}
14262025-03-25T23:01:12.055ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14272025-03-25T23:01:12.057ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
14282025-03-25T23:01:12.057ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
14292025-03-25T23:01:12.057ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
14302025-03-25T23:01:12.057ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
14312025-03-25T23:01:12.058ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 9af3e790-030d-4974-9b9e-c35f829f8643, lookup_type: ById(9af3e790-030d-4974-9b9e-c35f829f8643) } result = Ok(())
14322025-03-25T23:01:12.058ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
14332025-03-25T23:01:12.058ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
14342025-03-25T23:01:12.059ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14352025-03-25T23:01:12.059ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14362025-03-25T23:01:12.060ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 62.98466ms iteration = 24 status = 0
14372025-03-25T23:01:12.062ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
14382025-03-25T23:01:12.062ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
14392025-03-25T23:01:12.062ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
14402025-03-25T23:01:12.063ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 5d9d6f43-fd71-4256-b06d-1ed4c87e191f, lookup_type: ById(5d9d6f43-fd71-4256-b06d-1ed4c87e191f) } result = Ok(())
14412025-03-25T23:01:12.063ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
14422025-03-25T23:01:12.064ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
14432025-03-25T23:01:12.064ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
14442025-03-25T23:01:12.064ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 67.358569ms iteration = 24 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
14452025-03-25T23:01:12.064ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
14462025-03-25T23:01:12.065ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
14472025-03-25T23:01:12.066ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14482025-03-25T23:01:12.068ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
14492025-03-25T23:01:12.068ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
14502025-03-25T23:01:12.068ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
14512025-03-25T23:01:12.070ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:360
14522025-03-25T23:01:12.070ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 72.840732ms iteration = 24 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
14532025-03-25T23:01:12.073ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 5d9d6f43-fd71-4256-b06d-1ed4c87e191f original_time_created = 2025-03-25 22:49:35.383803 UTC target_id = 5d9d6f43-fd71-4256-b06d-1ed4c87e191f time_created = 2025-03-25 22:49:35.383803 UTC
14542025-03-25T23:01:12.073ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 77.076518ms iteration = 70 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"5d9d6f43-fd71-4256-b06d-1ed4c87e191f","time_created":"2025-03-25T22:49:35.383803Z"}
14552025-03-25T23:01:12.080ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 83.350679ms iteration = 24 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
14562025-03-25T23:01:12.082ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
14572025-03-25T23:01:12.082ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
14582025-03-25T23:01:12.088ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
14592025-03-25T23:01:12.088ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
14602025-03-25T23:01:12.088ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 91.264907ms iteration = 24 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
14612025-03-25T23:01:12.089ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1085 zone_address = fd00:1122:3344:101::2
14622025-03-25T23:01:12.089ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
14632025-03-25T23:01:12.089ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "b94317ee-2533-4ab8-a9f7-6a9bed5aa1bc", "content-length": "26", "date": "Tue, 25 Mar 2025 23:01:12 GMT"} })
14642025-03-25T23:01:12.089ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1088 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
14652025-03-25T23:01:12.089ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1085 zone_address = fd00:1122:3344:101::2
14662025-03-25T23:01:12.089ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
14672025-03-25T23:01:12.090ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "19950f1d-5433-43eb-af48-d1c165bca9c2", "content-length": "26", "date": "Tue, 25 Mar 2025 23:01:12 GMT"} })
14682025-03-25T23:01:12.090ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1088 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
14692025-03-25T23:01:12.091ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1122 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
14702025-03-25T23:01:12.092ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1122 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
14712025-03-25T23:01:12.117ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 119.880333ms iteration = 24 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
14722025-03-25T23:01:12.119ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1085 rack_id = 3ef3cc6b-3ffd-47a3-99ef-c57b25776619 zone_address = fd00:1122:3344:101::2
14732025-03-25T23:01:12.119ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 3ef3cc6b-3ffd-47a3-99ef-c57b25776619 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
14742025-03-25T23:01:12.120ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 3ef3cc6b-3ffd-47a3-99ef-c57b25776619 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "2ac3291f-d996-48ec-a637-e05e37bb8222", "content-length": "26", "date": "Tue, 25 Mar 2025 23:01:12 GMT"} })
14752025-03-25T23:01:12.120ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1088 rack_id = 3ef3cc6b-3ffd-47a3-99ef-c57b25776619 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
14762025-03-25T23:01:12.121ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1122 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 3ef3cc6b-3ffd-47a3-99ef-c57b25776619
14772025-03-25T23:01:12.128ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
14782025-03-25T23:01:12.128ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
14792025-03-25T23:01:12.130ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14802025-03-25T23:01:12.132ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
14812025-03-25T23:01:12.132ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
14822025-03-25T23:01:12.132ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
14832025-03-25T23:01:12.196ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 198.965485ms iteration = 27 status = {"physical_disks_added":0}
14842025-03-25T23:01:12.230ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
14852025-03-25T23:01:12.231ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "d52ba7f3-ff1e-4024-a7ca-40529998f277", "content-length": "2", "date": "Tue, 25 Mar 2025 23:01:12 GMT"} })
14862025-03-25T23:01:12.234ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
14872025-03-25T23:01:12.234ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
14882025-03-25T23:01:12.239ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 243.721204ms iteration = 24 status = {}
14892025-03-25T23:01:12.294ZINFOnexus (BackgroundTasks): VPC route manager sled 9af3e790-030d-4974-9b9e-c35f829f8643 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
14902025-03-25T23:01:12.294ZDEBGnexus (BackgroundTasks): client request SledAgent = 9af3e790-030d-4974-9b9e-c35f829f8643 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
14912025-03-25T23:01:12.294ZDEBGnexus (BackgroundTasks): client response SledAgent = 9af3e790-030d-4974-9b9e-c35f829f8643 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "809ecd61-39dc-4f1b-a769-27b37cdb5ba3", "content-length": "372", "date": "Tue, 25 Mar 2025 23:01:12 GMT"} })
14922025-03-25T23:01:12.294ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
14932025-03-25T23:01:12.295ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
14942025-03-25T23:01:12.297ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 301.23484ms iteration = 24 status = null
14952025-03-25T23:01:12.336ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14962025-03-25T23:01:12.339ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
14972025-03-25T23:01:12.339ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
14982025-03-25T23:01:12.339ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
14992025-03-25T23:01:12.355ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = buskin
15002025-03-25T23:01:12.355ZDEBGnexus (BackgroundTasks): client request SledAgent = 9af3e790-030d-4974-9b9e-c35f829f8643 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
15012025-03-25T23:01:12.355ZDEBGnexus (BackgroundTasks): client response SledAgent = 9af3e790-030d-4974-9b9e-c35f829f8643 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "7a3b971c-5b38-4e10-9291-f8a90fe206ef", "date": "Tue, 25 Mar 2025 23:01:12 GMT"} })
15022025-03-25T23:01:12.355ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
15032025-03-25T23:01:12.356ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
15042025-03-25T23:01:12.358ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
15052025-03-25T23:01:12.359ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
15062025-03-25T23:01:12.362ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
15072025-03-25T23:01:12.362ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
15082025-03-25T23:01:12.365ZINFOnexus (BackgroundTasks): VPC route manager sled 9af3e790-030d-4974-9b9e-c35f829f8643 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
15092025-03-25T23:01:12.365ZDEBGnexus (BackgroundTasks): client request SledAgent = 9af3e790-030d-4974-9b9e-c35f829f8643 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
15102025-03-25T23:01:12.366ZDEBGnexus (BackgroundTasks): client response SledAgent = 9af3e790-030d-4974-9b9e-c35f829f8643 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "0939c32b-4b93-45fa-bed6-261d4f1959d2", "date": "Tue, 25 Mar 2025 23:01:12 GMT"} })
15112025-03-25T23:01:12.367ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 370.386959ms iteration = 25 status = {}
15122025-03-25T23:01:12.385ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
15132025-03-25T23:01:12.385ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
15142025-03-25T23:01:12.388ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: 25f1877d-a731-48fe-8967-16ef2fb00f3b, rack_id: 3ef3cc6b-3ffd-47a3-99ef-c57b25776619, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(dcb3f9bf-5ea7-47e3-b699-6c3a8c7459c5) } rack_id = 3ef3cc6b-3ffd-47a3-99ef-c57b25776619 switch_location = Switch0
15152025-03-25T23:01:12.388ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
15162025-03-25T23:01:12.388ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
15172025-03-25T23:01:12.397ZDEBGnexus (BackgroundTasks): client request SledAgent = 9af3e790-030d-4974-9b9e-c35f829f8643 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
15182025-03-25T23:01:12.397ZDEBGnexus (BackgroundTasks): client response SledAgent = 9af3e790-030d-4974-9b9e-c35f829f8643 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "55c543e2-bd05-4ad1-a7fa-6ceb002268f3", "content-length": "691", "date": "Tue, 25 Mar 2025 23:01:12 GMT"} })
15192025-03-25T23:01:12.397ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = buskin
15202025-03-25T23:01:12.397ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = buskin
15212025-03-25T23:01:12.397ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = buskin
15222025-03-25T23:01:12.399ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 402.243548ms iteration = 25 status = {}
15232025-03-25T23:01:12.408ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 3ef3cc6b-3ffd-47a3-99ef-c57b25776619 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
15242025-03-25T23:01:12.409ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 3ef3cc6b-3ffd-47a3-99ef-c57b25776619 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "7fe8f4a1-d027-4e5d-8275-c880f3b19eed", "content-length": "134", "date": "Tue, 25 Mar 2025 23:01:12 GMT"} })
15252025-03-25T23:01:12.409ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1741 rack_id = 3ef3cc6b-3ffd-47a3-99ef-c57b25776619 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
15262025-03-25T23:01:12.409ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1768 rack_id = 3ef3cc6b-3ffd-47a3-99ef-c57b25776619 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = dcb3f9bf-5ea7-47e3-b699-6c3a8c7459c5
15272025-03-25T23:01:12.409ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:378 rack_id = 3ef3cc6b-3ffd-47a3-99ef-c57b25776619 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
15282025-03-25T23:01:12.409ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:382 rack_id = 3ef3cc6b-3ffd-47a3-99ef-c57b25776619 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
15292025-03-25T23:01:12.409ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:391 rack_id = 3ef3cc6b-3ffd-47a3-99ef-c57b25776619 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
15302025-03-25T23:01:12.409ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:397 rack_id = 3ef3cc6b-3ffd-47a3-99ef-c57b25776619 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
15312025-03-25T23:01:12.409ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = 3ef3cc6b-3ffd-47a3-99ef-c57b25776619 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
15322025-03-25T23:01:12.409ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1915 rack_id = 3ef3cc6b-3ffd-47a3-99ef-c57b25776619 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
15332025-03-25T23:01:12.409ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:408 rack_id = 3ef3cc6b-3ffd-47a3-99ef-c57b25776619 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
15342025-03-25T23:01:12.409ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1951 rack_id = 3ef3cc6b-3ffd-47a3-99ef-c57b25776619 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
15352025-03-25T23:01:12.409ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:417 rack_id = 3ef3cc6b-3ffd-47a3-99ef-c57b25776619
15362025-03-25T23:01:12.409ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
15372025-03-25T23:01:12.410ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
15382025-03-25T23:01:12.412ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 3ef3cc6b-3ffd-47a3-99ef-c57b25776619 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
15392025-03-25T23:01:12.412ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 3ef3cc6b-3ffd-47a3-99ef-c57b25776619 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "659e4591-ec4c-4444-bc31-fccf38e79786", "content-length": "2", "date": "Tue, 25 Mar 2025 23:01:12 GMT"} })
15402025-03-25T23:01:12.412ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 3ef3cc6b-3ffd-47a3-99ef-c57b25776619 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
15412025-03-25T23:01:12.412ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 3ef3cc6b-3ffd-47a3-99ef-c57b25776619 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "be3fb541-fc83-4ce7-9252-c59b7f3e7c71", "content-length": "52", "date": "Tue, 25 Mar 2025 23:01:12 GMT"} })
15422025-03-25T23:01:12.412ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.1.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:466 rack_id = 3ef3cc6b-3ffd-47a3-99ef-c57b25776619 switch_location = Switch0
15432025-03-25T23:01:12.412ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 3ef3cc6b-3ffd-47a3-99ef-c57b25776619 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
15442025-03-25T23:01:12.530ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 3ef3cc6b-3ffd-47a3-99ef-c57b25776619 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "8fdb2b06-bcfe-4743-93ea-4b126bc4af8e", "date": "Tue, 25 Mar 2025 23:01:12 GMT"} })
15452025-03-25T23:01:12.530ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:769 rack_id = 3ef3cc6b-3ffd-47a3-99ef-c57b25776619 switch = Switch0
15462025-03-25T23:01:12.530ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 3ef3cc6b-3ffd-47a3-99ef-c57b25776619 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
15472025-03-25T23:01:12.530ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 3ef3cc6b-3ffd-47a3-99ef-c57b25776619 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "fc39ae26-8fa4-488f-bf38-79d84f2ffc59", "content-length": "682", "date": "Tue, 25 Mar 2025 23:01:12 GMT"} })
15482025-03-25T23:01:12.530ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
15492025-03-25T23:01:12.531ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
15502025-03-25T23:01:12.534ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
15512025-03-25T23:01:12.535ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
15522025-03-25T23:01:12.538ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
15532025-03-25T23:01:12.538ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
15542025-03-25T23:01:12.540ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
15552025-03-25T23:01:12.540ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
15562025-03-25T23:01:12.548ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 551.012491ms iteration = 24 status = {}
15572025-03-25T23:01:13.072ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 754bbc75-9090-42a0-81c9-f1bb08a7e5ea roles = RoleSet { roles: {} } uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
15582025-03-25T23:01:13.073ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 754bbc75-9090-42a0-81c9-f1bb08a7e5ea resource = Database result = Ok(()) uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
15592025-03-25T23:01:13.075ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2399 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 754bbc75-9090-42a0-81c9-f1bb08a7e5ea response_code = 200 uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
15602025-03-25T23:01:13.856ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 655 local_addr = [fd00:1122:3344:101::a]:62174 method = GET remote_addr = [fd00:1122:3344:101::d]:63115 req_id = ce3535f7-c02d-4973-811b-718d85bb48ce response_code = 200 uri = /8f96aade-61ab-4710-a1b1-0efefa6704ee
15612025-03-25T23:01:14.094ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = f58c2c10-d494-45f8-a177-8b2c2a81249a roles = RoleSet { roles: {} } uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
15622025-03-25T23:01:14.095ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = f58c2c10-d494-45f8-a177-8b2c2a81249a resource = Database result = Ok(()) uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
15632025-03-25T23:01:14.097ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2247 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = f58c2c10-d494-45f8-a177-8b2c2a81249a response_code = 200 uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
15642025-03-25T23:01:15.117ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 3d384170-ec0b-4348-b374-ca00b19a72b3 roles = RoleSet { roles: {} } uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
15652025-03-25T23:01:15.118ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 3d384170-ec0b-4348-b374-ca00b19a72b3 resource = Database result = Ok(()) uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
15662025-03-25T23:01:15.120ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2921 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 3d384170-ec0b-4348-b374-ca00b19a72b3 response_code = 200 uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
15672025-03-25T23:01:16.141ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 940c599d-8f3a-4bdc-8c3e-dabaedb65d88 roles = RoleSet { roles: {} } uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
15682025-03-25T23:01:16.142ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 940c599d-8f3a-4bdc-8c3e-dabaedb65d88 resource = Database result = Ok(()) uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
15692025-03-25T23:01:16.144ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2688 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 940c599d-8f3a-4bdc-8c3e-dabaedb65d88 response_code = 200 uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
15702025-03-25T23:01:17.165ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 6cb4b467-ce5f-4d70-8d7f-9f7190926838 roles = RoleSet { roles: {} } uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
15712025-03-25T23:01:17.165ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 6cb4b467-ce5f-4d70-8d7f-9f7190926838 resource = Database result = Ok(()) uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
15722025-03-25T23:01:17.167ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2613 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 6cb4b467-ce5f-4d70-8d7f-9f7190926838 response_code = 200 uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
15732025-03-25T23:01:18.187ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = a2137d7f-6555-4b7c-a6f1-2adf541ff9ab roles = RoleSet { roles: {} } uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
15742025-03-25T23:01:18.187ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = a2137d7f-6555-4b7c-a6f1-2adf541ff9ab resource = Database result = Ok(()) uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
15752025-03-25T23:01:18.189ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2311 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = a2137d7f-6555-4b7c-a6f1-2adf541ff9ab response_code = 200 uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
15762025-03-25T23:01:19.209ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 8ddbabee-bcf0-46fb-8a50-55e8b255a26c roles = RoleSet { roles: {} } uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
15772025-03-25T23:01:19.210ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 8ddbabee-bcf0-46fb-8a50-55e8b255a26c resource = Database result = Ok(()) uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
15782025-03-25T23:01:19.211ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1955 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 8ddbabee-bcf0-46fb-8a50-55e8b255a26c response_code = 200 uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
15792025-03-25T23:01:20.231ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 07e8ffbc-0980-4349-94a2-3d6562de4658 roles = RoleSet { roles: {} } uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
15802025-03-25T23:01:20.232ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 07e8ffbc-0980-4349-94a2-3d6562de4658 resource = Database result = Ok(()) uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
15812025-03-25T23:01:20.234ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2460 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 07e8ffbc-0980-4349-94a2-3d6562de4658 response_code = 200 uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
15822025-03-25T23:01:21.255ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = c38c5eae-f762-4a40-a73d-af73e966755e roles = RoleSet { roles: {} } uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
15832025-03-25T23:01:21.255ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = c38c5eae-f762-4a40-a73d-af73e966755e resource = Database result = Ok(()) uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
15842025-03-25T23:01:21.258ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2590 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = c38c5eae-f762-4a40-a73d-af73e966755e response_code = 200 uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
15852025-03-25T23:01:21.996ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 71 reason = Timeout
15862025-03-25T23:01:21.996ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
15872025-03-25T23:01:21.996ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
15882025-03-25T23:01:21.996ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 71 reason = Timeout
15892025-03-25T23:01:21.997ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
15902025-03-25T23:01:21.997ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
15912025-03-25T23:01:21.999ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
15922025-03-25T23:01:21.999ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15932025-03-25T23:01:21.999ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
15942025-03-25T23:01:22.002ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
15952025-03-25T23:01:22.002ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
15962025-03-25T23:01:22.002ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
15972025-03-25T23:01:22.004ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
15982025-03-25T23:01:22.004ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
15992025-03-25T23:01:22.006ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16002025-03-25T23:01:22.008ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 5d9d6f43-fd71-4256-b06d-1ed4c87e191f, lookup_type: ById(5d9d6f43-fd71-4256-b06d-1ed4c87e191f) } result = Ok(())
16012025-03-25T23:01:22.008ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
16022025-03-25T23:01:22.009ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
16032025-03-25T23:01:22.016ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 5d9d6f43-fd71-4256-b06d-1ed4c87e191f original_time_created = 2025-03-25 22:49:35.383803 UTC target_id = 5d9d6f43-fd71-4256-b06d-1ed4c87e191f time_created = 2025-03-25 22:49:35.383803 UTC
16042025-03-25T23:01:22.016ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 19.912729ms iteration = 71 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"5d9d6f43-fd71-4256-b06d-1ed4c87e191f","time_created":"2025-03-25T22:49:35.383803Z"}
16052025-03-25T23:01:22.032ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 35.0162ms iteration = 71 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
16062025-03-25T23:01:22.278ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 558df741-71b5-471e-832a-d7808cd84584 roles = RoleSet { roles: {} } uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
16072025-03-25T23:01:22.279ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 558df741-71b5-471e-832a-d7808cd84584 resource = Database result = Ok(()) uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
16082025-03-25T23:01:22.280ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1678 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 558df741-71b5-471e-832a-d7808cd84584 response_code = 200 uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
16092025-03-25T23:01:23.301ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 8103bd32-d505-4512-8747-e20e24d3e7d9 roles = RoleSet { roles: {} } uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
16102025-03-25T23:01:23.301ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 8103bd32-d505-4512-8747-e20e24d3e7d9 resource = Database result = Ok(()) uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
16112025-03-25T23:01:23.303ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2179 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 8103bd32-d505-4512-8747-e20e24d3e7d9 response_code = 200 uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
16122025-03-25T23:01:23.856ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 580 local_addr = [fd00:1122:3344:101::a]:62174 method = GET remote_addr = [fd00:1122:3344:101::d]:63115 req_id = 53a55747-aabe-4ccd-9439-ce6e89e63b65 response_code = 200 uri = /8f96aade-61ab-4710-a1b1-0efefa6704ee
16132025-03-25T23:01:24.323ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = ecd5fea6-125d-4c37-85d3-e4b78c91b386 roles = RoleSet { roles: {} } uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
16142025-03-25T23:01:24.323ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = ecd5fea6-125d-4c37-85d3-e4b78c91b386 resource = Database result = Ok(()) uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
16152025-03-25T23:01:24.325ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2599 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = ecd5fea6-125d-4c37-85d3-e4b78c91b386 response_code = 200 uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
16162025-03-25T23:01:25.346ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 4376d4ab-f9f8-4295-a369-8287d7bdf9cb roles = RoleSet { roles: {} } uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
16172025-03-25T23:01:25.346ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 4376d4ab-f9f8-4295-a369-8287d7bdf9cb resource = Database result = Ok(()) uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
16182025-03-25T23:01:25.348ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2735 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 4376d4ab-f9f8-4295-a369-8287d7bdf9cb response_code = 200 uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
16192025-03-25T23:01:26.368ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = f543d071-33f1-4e35-8612-e0ec21eeedab roles = RoleSet { roles: {} } uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
16202025-03-25T23:01:26.369ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = f543d071-33f1-4e35-8612-e0ec21eeedab resource = Database result = Ok(()) uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
16212025-03-25T23:01:26.371ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2521 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = f543d071-33f1-4e35-8612-e0ec21eeedab response_code = 200 uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
16222025-03-25T23:01:27.392ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 66f719e2-0f5e-451e-bf3b-e5bb6679fac9 roles = RoleSet { roles: {} } uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
16232025-03-25T23:01:27.392ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 66f719e2-0f5e-451e-bf3b-e5bb6679fac9 resource = Database result = Ok(()) uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
16242025-03-25T23:01:27.394ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2301 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 66f719e2-0f5e-451e-bf3b-e5bb6679fac9 response_code = 200 uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
16252025-03-25T23:01:28.414ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = d260e7e6-c7d2-4455-aecf-66ccdce1fc67 roles = RoleSet { roles: {} } uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
16262025-03-25T23:01:28.414ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = d260e7e6-c7d2-4455-aecf-66ccdce1fc67 resource = Database result = Ok(()) uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
16272025-03-25T23:01:28.416ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2205 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = d260e7e6-c7d2-4455-aecf-66ccdce1fc67 response_code = 200 uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
16282025-03-25T23:01:29.437ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = f5cbf7eb-597a-4397-a7dc-ecb8f51eda54 roles = RoleSet { roles: {} } uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
16292025-03-25T23:01:29.438ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = f5cbf7eb-597a-4397-a7dc-ecb8f51eda54 resource = Database result = Ok(()) uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
16302025-03-25T23:01:29.439ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2165 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = f5cbf7eb-597a-4397-a7dc-ecb8f51eda54 response_code = 200 uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
16312025-03-25T23:01:30.462ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 7f2962b0-4c6b-47e4-acd1-139de39d204f roles = RoleSet { roles: {} } uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
16322025-03-25T23:01:30.462ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 7f2962b0-4c6b-47e4-acd1-139de39d204f resource = Database result = Ok(()) uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
16332025-03-25T23:01:30.464ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2438 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = 7f2962b0-4c6b-47e4-acd1-139de39d204f response_code = 200 uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
16342025-03-25T23:01:31.485ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = fcb8217d-775b-476a-a6ea-26dbf98c3ef4 roles = RoleSet { roles: {} } uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
16352025-03-25T23:01:31.486ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = fcb8217d-775b-476a-a6ea-26dbf98c3ef4 resource = Database result = Ok(()) uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643
16362025-03-25T23:01:31.487ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2197 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:52197 req_id = fcb8217d-775b-476a-a6ea-26dbf98c3ef4 response_code = 200 uri = /probes/9af3e790-030d-4974-9b9e-c35f829f8643