Max level shown:
12024-12-14T01:15:01.261ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = fa8155b7-735f-447b-9a61-e70026f17138 roles = RoleSet { roles: {} } uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
22024-12-14T01:15:01.262ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = fa8155b7-735f-447b-9a61-e70026f17138 resource = Database result = Ok(()) uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
32024-12-14T01:15:01.265ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 3938 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = fa8155b7-735f-447b-9a61-e70026f17138 response_code = 200 uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
42024-12-14T01:15:02.295ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 38eb94ba-f441-47b2-ac0e-9b77221e060e roles = RoleSet { roles: {} } uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
52024-12-14T01:15:02.295ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 38eb94ba-f441-47b2-ac0e-9b77221e060e resource = Database result = Ok(()) uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
62024-12-14T01:15:02.298ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2796 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 38eb94ba-f441-47b2-ac0e-9b77221e060e response_code = 200 uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
72024-12-14T01:15:03.330ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 1be8f2cd-f1e0-45b3-b415-b46a96d367b9 roles = RoleSet { roles: {} } uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
82024-12-14T01:15:03.330ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 1be8f2cd-f1e0-45b3-b415-b46a96d367b9 resource = Database result = Ok(()) uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
92024-12-14T01:15:03.333ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 3031 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 1be8f2cd-f1e0-45b3-b415-b46a96d367b9 response_code = 200 uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
102024-12-14T01:15:03.452ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:57045 req_id = b06edc83-4c5b-4b94-8b8e-a3640d70256d roles = RoleSet { roles: {} } uri = /metrics/collectors/6e1e4a25-c6a7-49c5-bb1d-86638130bef7/producers?sort_by=id_ascending
112024-12-14T01:15:03.452ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:57045 req_id = b06edc83-4c5b-4b94-8b8e-a3640d70256d resource = Database result = Ok(()) uri = /metrics/collectors/6e1e4a25-c6a7-49c5-bb1d-86638130bef7/producers?sort_by=id_ascending
122024-12-14T01:15:03.455ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 3671 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:57045 req_id = b06edc83-4c5b-4b94-8b8e-a3640d70256d response_code = 200 uri = /metrics/collectors/6e1e4a25-c6a7-49c5-bb1d-86638130bef7/producers?sort_by=id_ascending
132024-12-14T01:15:03.455ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:57045 req_id = 4cffb0ae-c98c-41a4-8daa-2cc5d71150ef roles = RoleSet { roles: {} } uri = /metrics/collectors/6e1e4a25-c6a7-49c5-bb1d-86638130bef7/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmNGM2ZGNlZi0yNTc3LTQxMmUtODNkMS0zNzkyZGNjMDQ2MWQifX0%3D
142024-12-14T01:15:03.456ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:57045 req_id = 4cffb0ae-c98c-41a4-8daa-2cc5d71150ef resource = Database result = Ok(()) uri = /metrics/collectors/6e1e4a25-c6a7-49c5-bb1d-86638130bef7/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmNGM2ZGNlZi0yNTc3LTQxMmUtODNkMS0zNzkyZGNjMDQ2MWQifX0%3D
152024-12-14T01:15:03.458ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 3064 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:57045 req_id = 4cffb0ae-c98c-41a4-8daa-2cc5d71150ef response_code = 200 uri = /metrics/collectors/6e1e4a25-c6a7-49c5-bb1d-86638130bef7/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmNGM2ZGNlZi0yNTc3LTQxMmUtODNkMS0zNzkyZGNjMDQ2MWQifX0%3D
162024-12-14T01:15:03.462ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1259 local_addr = [fd00:1122:3344:101::c]:54941 method = GET remote_addr = [fd00:1122:3344:101::d]:43526 req_id = fb1cc30b-3e49-4642-8ea9-5ea0541dbde9 response_code = 200 uri = /499bc009-7d23-4916-9be8-0a6a8f4e2f93
172024-12-14T01:15:04.358ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = d2f9e402-92ce-42fd-a7af-826f7195458b roles = RoleSet { roles: {} } uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
182024-12-14T01:15:04.358ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = d2f9e402-92ce-42fd-a7af-826f7195458b resource = Database result = Ok(()) uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
192024-12-14T01:15:04.360ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2566 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = d2f9e402-92ce-42fd-a7af-826f7195458b response_code = 200 uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
202024-12-14T01:15:04.747ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 71 reason = Timeout
212024-12-14T01:15:04.747ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 71 reason = Timeout
222024-12-14T01:15:04.747ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
232024-12-14T01:15:04.747ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
242024-12-14T01:15:04.747ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
252024-12-14T01:15:04.747ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
262024-12-14T01:15:04.749ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
272024-12-14T01:15:04.751ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
282024-12-14T01:15:04.751ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
292024-12-14T01:15:04.751ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
302024-12-14T01:15:04.754ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
312024-12-14T01:15:04.754ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
322024-12-14T01:15:04.777ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
332024-12-14T01:15:04.777ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
342024-12-14T01:15:04.780ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
352024-12-14T01:15:04.783ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 8770bfcc-3a71-4738-a6b0-91008cadcb0f, lookup_type: ById(8770bfcc-3a71-4738-a6b0-91008cadcb0f) } result = Ok(())
362024-12-14T01:15:04.783ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
372024-12-14T01:15:04.783ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
382024-12-14T01:15:04.795ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 8770bfcc-3a71-4738-a6b0-91008cadcb0f original_time_created = 2024-12-14 01:03:15.378067 UTC target_id = 8770bfcc-3a71-4738-a6b0-91008cadcb0f time_created = 2024-12-14 01:03:15.378067 UTC
392024-12-14T01:15:04.795ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 48.360569ms iteration = 71 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"8770bfcc-3a71-4738-a6b0-91008cadcb0f","time_created":"2024-12-14T01:03:15.378067Z"}
402024-12-14T01:15:04.805ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 58.509003ms iteration = 71 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
412024-12-14T01:15:05.380ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 37665d48-6a7f-4ff9-812c-1cda0e0f0d22 roles = RoleSet { roles: {} } uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
422024-12-14T01:15:05.380ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 37665d48-6a7f-4ff9-812c-1cda0e0f0d22 resource = Database result = Ok(()) uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
432024-12-14T01:15:05.383ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2778 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 37665d48-6a7f-4ff9-812c-1cda0e0f0d22 response_code = 200 uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
442024-12-14T01:15:06.405ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 97e35b7a-3ff9-49c4-bd33-e613251f9ebc roles = RoleSet { roles: {} } uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
452024-12-14T01:15:06.405ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 97e35b7a-3ff9-49c4-bd33-e613251f9ebc resource = Database result = Ok(()) uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
462024-12-14T01:15:06.407ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2360 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 97e35b7a-3ff9-49c4-bd33-e613251f9ebc response_code = 200 uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
472024-12-14T01:15:07.080ZINFOnexus (dropshot_internal): received new VMM runtime state from sled agent actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true file = nexus/src/app/instance.rs:2116 local_addr = [fd00:1122:3344:101::c]:12221 method = PUT migration_state = Migrations { migration_in: None, migration_out: None } propolis_id = b2d035ea-f96d-499b-a259-358e56f0523e remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 88f174f9-f0d7-4b3a-b2ca-5f8538ca40dc uri = /vmms/b2d035ea-f96d-499b-a259-358e56f0523e vmm_state = VmmRuntimeState { state: Starting, gen: Generation(3), time_updated: 2024-12-14T01:15:07.069606756Z }
482024-12-14T01:15:07.080ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 88f174f9-f0d7-4b3a-b2ca-5f8538ca40dc roles = RoleSet { roles: {} } uri = /vmms/b2d035ea-f96d-499b-a259-358e56f0523e
492024-12-14T01:15:07.080ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 88f174f9-f0d7-4b3a-b2ca-5f8538ca40dc resource = Database result = Ok(()) uri = /vmms/b2d035ea-f96d-499b-a259-358e56f0523e
502024-12-14T01:15:07.103ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::c]:12221 remote_addr = [fd00:1122:3344:101::1:0]:39505
512024-12-14T01:15:07.104ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::1:0]:39505 req_id = 0bdc1ab6-964e-4a9f-868e-85b3a0c52977 roles = RoleSet { roles: {} } uri = /metrics/producers
522024-12-14T01:15:07.104ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::1:0]:39505 req_id = 0bdc1ab6-964e-4a9f-868e-85b3a0c52977 resource = Database result = Ok(()) uri = /metrics/producers
532024-12-14T01:15:07.121ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 51735 local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 88f174f9-f0d7-4b3a-b2ca-5f8538ca40dc response_code = 204 uri = /vmms/b2d035ea-f96d-499b-a259-358e56f0523e
542024-12-14T01:15:07.122ZINFOnexus (dropshot_internal): received new VMM runtime state from sled agent actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true file = nexus/src/app/instance.rs:2116 local_addr = [fd00:1122:3344:101::c]:12221 method = PUT migration_state = Migrations { migration_in: None, migration_out: None } propolis_id = b2d035ea-f96d-499b-a259-358e56f0523e remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 585da241-ae5b-4aaa-91da-623506959b99 uri = /vmms/b2d035ea-f96d-499b-a259-358e56f0523e vmm_state = VmmRuntimeState { state: Running, gen: Generation(4), time_updated: 2024-12-14T01:15:07.122032673Z }
552024-12-14T01:15:07.122ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 585da241-ae5b-4aaa-91da-623506959b99 roles = RoleSet { roles: {} } uri = /vmms/b2d035ea-f96d-499b-a259-358e56f0523e
562024-12-14T01:15:07.122ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 585da241-ae5b-4aaa-91da-623506959b99 resource = Database result = Ok(()) uri = /vmms/b2d035ea-f96d-499b-a259-358e56f0523e
572024-12-14T01:15:07.140ZINFOnexus: assigned collector to new producer collector_id = 6e1e4a25-c6a7-49c5-bb1d-86638130bef7 file = nexus/src/app/oximeter.rs:76 producer_id = d008f4b2-6063-424c-a333-d54d35c8c058
582024-12-14T01:15:07.140ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 36231 local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::1:0]:39505 req_id = 0bdc1ab6-964e-4a9f-868e-85b3a0c52977 response_code = 201 uri = /metrics/producers
592024-12-14T01:15:07.157ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 35774 local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 585da241-ae5b-4aaa-91da-623506959b99 response_code = 204 uri = /vmms/b2d035ea-f96d-499b-a259-358e56f0523e
602024-12-14T01:15:07.427ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = d40983d2-41c7-49b0-8b95-c1a052a6a52f roles = RoleSet { roles: {} } uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
612024-12-14T01:15:07.428ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = d40983d2-41c7-49b0-8b95-c1a052a6a52f resource = Database result = Ok(()) uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
622024-12-14T01:15:07.430ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2735 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = d40983d2-41c7-49b0-8b95-c1a052a6a52f response_code = 200 uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
632024-12-14T01:15:08.450ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 07d974ea-3584-4e09-9a9e-e73bd55bbb36 roles = RoleSet { roles: {} } uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
642024-12-14T01:15:08.451ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 07d974ea-3584-4e09-9a9e-e73bd55bbb36 resource = Database result = Ok(()) uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
652024-12-14T01:15:08.453ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2371 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 07d974ea-3584-4e09-9a9e-e73bd55bbb36 response_code = 200 uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
662024-12-14T01:15:09.472ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 3fc811e6-1b30-4dc0-9e8a-ba2da68a3be5 roles = RoleSet { roles: {} } uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
672024-12-14T01:15:09.473ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 3fc811e6-1b30-4dc0-9e8a-ba2da68a3be5 resource = Database result = Ok(()) uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
682024-12-14T01:15:09.475ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2314 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 3fc811e6-1b30-4dc0-9e8a-ba2da68a3be5 response_code = 200 uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
692024-12-14T01:15:10.495ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 6df5e75a-9336-42fb-97d5-f2554022a35c roles = RoleSet { roles: {} } uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
702024-12-14T01:15:10.495ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 6df5e75a-9336-42fb-97d5-f2554022a35c resource = Database result = Ok(()) uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
712024-12-14T01:15:10.497ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2397 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 6df5e75a-9336-42fb-97d5-f2554022a35c response_code = 200 uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
722024-12-14T01:15:11.518ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 5cc8a1fb-9110-41fc-82e4-6695408b6f90 roles = RoleSet { roles: {} } uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
732024-12-14T01:15:11.518ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 5cc8a1fb-9110-41fc-82e4-6695408b6f90 resource = Database result = Ok(()) uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
742024-12-14T01:15:11.520ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2116 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 5cc8a1fb-9110-41fc-82e4-6695408b6f90 response_code = 200 uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
752024-12-14T01:15:12.539ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = edf3c25b-1d55-4b33-a62e-1b7f39b1e24d roles = RoleSet { roles: {} } uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
762024-12-14T01:15:12.540ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = edf3c25b-1d55-4b33-a62e-1b7f39b1e24d resource = Database result = Ok(()) uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
772024-12-14T01:15:12.542ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2332 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = edf3c25b-1d55-4b33-a62e-1b7f39b1e24d response_code = 200 uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
782024-12-14T01:15:13.462ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1498 local_addr = [fd00:1122:3344:101::c]:54941 method = GET remote_addr = [fd00:1122:3344:101::d]:43526 req_id = 9bd106cd-b156-4b2e-88d5-0af544064e4c response_code = 200 uri = /499bc009-7d23-4916-9be8-0a6a8f4e2f93
792024-12-14T01:15:13.562ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 565cd847-be9a-4240-8800-62881fd32b26 roles = RoleSet { roles: {} } uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
802024-12-14T01:15:13.562ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 565cd847-be9a-4240-8800-62881fd32b26 resource = Database result = Ok(()) uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
812024-12-14T01:15:13.564ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2386 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 565cd847-be9a-4240-8800-62881fd32b26 response_code = 200 uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
822024-12-14T01:15:14.585ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 17c4e89d-d4b6-48e9-824f-f617fdddf5a4 roles = RoleSet { roles: {} } uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
832024-12-14T01:15:14.586ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 17c4e89d-d4b6-48e9-824f-f617fdddf5a4 resource = Database result = Ok(()) uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
842024-12-14T01:15:14.587ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1992 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 17c4e89d-d4b6-48e9-824f-f617fdddf5a4 response_code = 200 uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
852024-12-14T01:15:14.746ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 72 reason = Timeout
862024-12-14T01:15:14.747ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 72 reason = Timeout
872024-12-14T01:15:14.747ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
882024-12-14T01:15:14.747ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
892024-12-14T01:15:14.747ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
902024-12-14T01:15:14.747ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
912024-12-14T01:15:14.748ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
922024-12-14T01:15:14.752ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
932024-12-14T01:15:14.752ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
942024-12-14T01:15:14.752ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
952024-12-14T01:15:14.753ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
962024-12-14T01:15:14.753ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
972024-12-14T01:15:14.754ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
982024-12-14T01:15:14.754ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
992024-12-14T01:15:14.755ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1002024-12-14T01:15:14.758ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 8770bfcc-3a71-4738-a6b0-91008cadcb0f, lookup_type: ById(8770bfcc-3a71-4738-a6b0-91008cadcb0f) } result = Ok(())
1012024-12-14T01:15:14.758ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
1022024-12-14T01:15:14.758ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
1032024-12-14T01:15:14.774ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 8770bfcc-3a71-4738-a6b0-91008cadcb0f original_time_created = 2024-12-14 01:03:15.378067 UTC target_id = 8770bfcc-3a71-4738-a6b0-91008cadcb0f time_created = 2024-12-14 01:03:15.378067 UTC
1042024-12-14T01:15:14.774ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 27.093895ms iteration = 72 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"8770bfcc-3a71-4738-a6b0-91008cadcb0f","time_created":"2024-12-14T01:03:15.378067Z"}
1052024-12-14T01:15:14.787ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 40.280466ms iteration = 72 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
1062024-12-14T01:15:15.607ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 05092250-7c17-44b6-acff-e88aafd930df roles = RoleSet { roles: {} } uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
1072024-12-14T01:15:15.607ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 05092250-7c17-44b6-acff-e88aafd930df resource = Database result = Ok(()) uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
1082024-12-14T01:15:15.609ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2168 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 05092250-7c17-44b6-acff-e88aafd930df response_code = 200 uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
1092024-12-14T01:15:15.803ZINFOnexus (dropshot_internal): received new VMM runtime state from sled agent actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true file = nexus/src/app/instance.rs:2116 local_addr = [fd00:1122:3344:101::c]:12221 method = PUT migration_state = Migrations { migration_in: None, migration_out: None } propolis_id = b2d035ea-f96d-499b-a259-358e56f0523e remote_addr = [fd00:1122:3344:101::1]:50234 req_id = fa9e40e5-8e62-43c8-ad86-b427b01163bd uri = /vmms/b2d035ea-f96d-499b-a259-358e56f0523e vmm_state = VmmRuntimeState { state: Stopping, gen: Generation(6), time_updated: 2024-12-14T01:15:15.802748421Z }
1102024-12-14T01:15:15.803ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:50234 req_id = fa9e40e5-8e62-43c8-ad86-b427b01163bd roles = RoleSet { roles: {} } uri = /vmms/b2d035ea-f96d-499b-a259-358e56f0523e
1112024-12-14T01:15:15.803ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:50234 req_id = fa9e40e5-8e62-43c8-ad86-b427b01163bd resource = Database result = Ok(()) uri = /vmms/b2d035ea-f96d-499b-a259-358e56f0523e
1122024-12-14T01:15:15.849ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 46726 local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:50234 req_id = fa9e40e5-8e62-43c8-ad86-b427b01163bd response_code = 204 uri = /vmms/b2d035ea-f96d-499b-a259-358e56f0523e
1132024-12-14T01:15:16.058ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:43741 req_id = c3d1cad3-c70d-4d0e-a8b0-c0f456d288b8 roles = RoleSet { roles: {} } uri = /nat/ipv4/changeset/8?limit=100
1142024-12-14T01:15:16.058ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:43741 req_id = c3d1cad3-c70d-4d0e-a8b0-c0f456d288b8 resource = Database result = Ok(()) uri = /nat/ipv4/changeset/8?limit=100
1152024-12-14T01:15:16.063ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 5463 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:43741 req_id = c3d1cad3-c70d-4d0e-a8b0-c0f456d288b8 response_code = 200 uri = /nat/ipv4/changeset/8?limit=100
1162024-12-14T01:15:16.064ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:43741 req_id = ee1117c2-56d5-4fa4-8b85-78b79cba8e03 roles = RoleSet { roles: {} } uri = /nat/ipv4/changeset/10?limit=100
1172024-12-14T01:15:16.064ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:43741 req_id = ee1117c2-56d5-4fa4-8b85-78b79cba8e03 resource = Database result = Ok(()) uri = /nat/ipv4/changeset/10?limit=100
1182024-12-14T01:15:16.067ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 3357 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:43741 req_id = ee1117c2-56d5-4fa4-8b85-78b79cba8e03 response_code = 200 uri = /nat/ipv4/changeset/10?limit=100
1192024-12-14T01:15:16.630ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 698063b2-f5b0-40a0-86a3-4982bce88627 roles = RoleSet { roles: {} } uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
1202024-12-14T01:15:16.630ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 698063b2-f5b0-40a0-86a3-4982bce88627 resource = Database result = Ok(()) uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
1212024-12-14T01:15:16.632ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2256 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 698063b2-f5b0-40a0-86a3-4982bce88627 response_code = 200 uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
1222024-12-14T01:15:17.649ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 5305b6fb-fc5e-4621-bb4e-8184e2db4d86 roles = RoleSet { roles: {} } uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
1232024-12-14T01:15:17.649ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 5305b6fb-fc5e-4621-bb4e-8184e2db4d86 resource = Database result = Ok(()) uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
1242024-12-14T01:15:17.651ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1623 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 5305b6fb-fc5e-4621-bb4e-8184e2db4d86 response_code = 200 uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
1252024-12-14T01:15:18.069ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:43741 req_id = 114ee20d-f3f7-45b1-bb16-de555a772c52 roles = RoleSet { roles: {} } uri = /nat/ipv4/changeset/10?limit=100
1262024-12-14T01:15:18.070ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:43741 req_id = 114ee20d-f3f7-45b1-bb16-de555a772c52 resource = Database result = Ok(()) uri = /nat/ipv4/changeset/10?limit=100
1272024-12-14T01:15:18.072ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 3335 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:43741 req_id = 114ee20d-f3f7-45b1-bb16-de555a772c52 response_code = 200 uri = /nat/ipv4/changeset/10?limit=100
1282024-12-14T01:15:18.279ZINFOnexus (dropshot_internal): received new VMM runtime state from sled agent actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true file = nexus/src/app/instance.rs:2116 local_addr = [fd00:1122:3344:101::c]:12221 method = PUT migration_state = Migrations { migration_in: None, migration_out: None } propolis_id = b2d035ea-f96d-499b-a259-358e56f0523e remote_addr = [fd00:1122:3344:101::1]:50234 req_id = ada2b5cd-960d-4809-972c-a3515d95f931 uri = /vmms/b2d035ea-f96d-499b-a259-358e56f0523e vmm_state = VmmRuntimeState { state: Destroyed, gen: Generation(8), time_updated: 2024-12-14T01:15:18.279588828Z }
1292024-12-14T01:15:18.279ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:50234 req_id = ada2b5cd-960d-4809-972c-a3515d95f931 roles = RoleSet { roles: {} } uri = /vmms/b2d035ea-f96d-499b-a259-358e56f0523e
1302024-12-14T01:15:18.280ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:50234 req_id = ada2b5cd-960d-4809-972c-a3515d95f931 resource = Database result = Ok(()) uri = /vmms/b2d035ea-f96d-499b-a259-358e56f0523e
1312024-12-14T01:15:18.315ZDEBGnexus (dropshot_internal): new VMM runtime state from sled agent requires an instance-update saga actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT migration_in_needs_update = false migration_out_needs_update = false propolis_id = b2d035ea-f96d-499b-a259-358e56f0523e remote_addr = [fd00:1122:3344:101::1]:50234 req_id = ada2b5cd-960d-4809-972c-a3515d95f931 uri = /vmms/b2d035ea-f96d-499b-a259-358e56f0523e vmm_needs_update = true
1322024-12-14T01:15:18.315ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:50234 req_id = ada2b5cd-960d-4809-972c-a3515d95f931 roles = RoleSet { roles: {} } uri = /vmms/b2d035ea-f96d-499b-a259-358e56f0523e
1332024-12-14T01:15:18.315ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:50234 req_id = ada2b5cd-960d-4809-972c-a3515d95f931 resource = Database result = Ok(()) uri = /vmms/b2d035ea-f96d-499b-a259-358e56f0523e
1342024-12-14T01:15:18.318ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:50234 req_id = ada2b5cd-960d-4809-972c-a3515d95f931 roles = RoleSet { roles: {} } uri = /vmms/b2d035ea-f96d-499b-a259-358e56f0523e
1352024-12-14T01:15:18.318ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:50234 req_id = ada2b5cd-960d-4809-972c-a3515d95f931 resource = Database result = Ok(()) uri = /vmms/b2d035ea-f96d-499b-a259-358e56f0523e
1362024-12-14T01:15:18.319ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:50234 req_id = ada2b5cd-960d-4809-972c-a3515d95f931 roles = RoleSet { roles: {} } uri = /vmms/b2d035ea-f96d-499b-a259-358e56f0523e
1372024-12-14T01:15:18.320ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:50234 req_id = ada2b5cd-960d-4809-972c-a3515d95f931 resource = Database result = Ok(()) uri = /vmms/b2d035ea-f96d-499b-a259-358e56f0523e
1382024-12-14T01:15:18.322ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:50234 req_id = ada2b5cd-960d-4809-972c-a3515d95f931 roles = RoleSet { roles: {} } uri = /vmms/b2d035ea-f96d-499b-a259-358e56f0523e
1392024-12-14T01:15:18.322ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:50234 req_id = ada2b5cd-960d-4809-972c-a3515d95f931 resource = Database result = Ok(()) uri = /vmms/b2d035ea-f96d-499b-a259-358e56f0523e
1402024-12-14T01:15:18.323ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:50234 req_id = ada2b5cd-960d-4809-972c-a3515d95f931 roles = RoleSet { roles: {} } uri = /vmms/b2d035ea-f96d-499b-a259-358e56f0523e
1412024-12-14T01:15:18.323ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:50234 req_id = ada2b5cd-960d-4809-972c-a3515d95f931 resource = Database result = Ok(()) uri = /vmms/b2d035ea-f96d-499b-a259-358e56f0523e
1422024-12-14T01:15:18.351ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:50234 req_id = ada2b5cd-960d-4809-972c-a3515d95f931 roles = RoleSet { roles: {} } uri = /vmms/b2d035ea-f96d-499b-a259-358e56f0523e
1432024-12-14T01:15:18.352ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:50234 req_id = ada2b5cd-960d-4809-972c-a3515d95f931 resource = Database result = Ok(()) uri = /vmms/b2d035ea-f96d-499b-a259-358e56f0523e
1442024-12-14T01:15:18.353ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:50234 req_id = ada2b5cd-960d-4809-972c-a3515d95f931 roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } uri = /vmms/b2d035ea-f96d-499b-a259-358e56f0523e
1452024-12-14T01:15:18.356ZDEBGnexus (dropshot_internal): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:50234 req_id = ada2b5cd-960d-4809-972c-a3515d95f931 resource = Instance { parent: Project { parent: Silo { parent: Fleet, key: 2c668072-a9a1-4a69-928c-e118c99521be, lookup_type: ById(2c668072-a9a1-4a69-928c-e118c99521be) }, key: 776ccd20-a853-4c93-af22-41ffed980f3e, lookup_type: ById(776ccd20-a853-4c93-af22-41ffed980f3e) }, key: d008f4b2-6063-424c-a333-d54d35c8c058, lookup_type: ById(d008f4b2-6063-424c-a333-d54d35c8c058) } result = Ok(()) uri = /vmms/b2d035ea-f96d-499b-a259-358e56f0523e
1462024-12-14T01:15:18.356ZINFOnexus (dropshot_internal): starting update saga for d008f4b2-6063-424c-a333-d54d35c8c058 actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true file = nexus/src/app/instance.rs:1583 instance_id = d008f4b2-6063-424c-a333-d54d35c8c058 local_addr = [fd00:1122:3344:101::c]:12221 method = PUT migration_state = Migrations { migration_in: None, migration_out: None } remote_addr = [fd00:1122:3344:101::1]:50234 req_id = ada2b5cd-960d-4809-972c-a3515d95f931 uri = /vmms/b2d035ea-f96d-499b-a259-358e56f0523e vmm_state = VmmRuntimeState { state: Destroyed, gen: Generation(8), time_updated: 2024-12-14T01:15:18.279588828Z }
1472024-12-14T01:15:18.356ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 76740 local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:50234 req_id = ada2b5cd-960d-4809-972c-a3515d95f931 response_code = 204 uri = /vmms/b2d035ea-f96d-499b-a259-358e56f0523e
1482024-12-14T01:15:18.356ZDEBGnexus (dropshot_internal): preparing instance-update saga for d008f4b2-6063-424c-a333-d54d35c8c058... actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true instance_id = d008f4b2-6063-424c-a333-d54d35c8c058 local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:50234 req_id = ada2b5cd-960d-4809-972c-a3515d95f931 uri = /vmms/b2d035ea-f96d-499b-a259-358e56f0523e
1492024-12-14T01:15:18.356ZINFOnexus (SagaExecutor): preparing saga file = nexus/src/app/saga.rs:255 saga_id = 0002aa10-b3d2-4ba2-8c02-cc3d7e716b42 saga_name = start-instance-update
1502024-12-14T01:15:18.356ZINFOnexus (SEC): saga create dag = {"end_node":4,"graph":{"edge_property":"directed","edges":[[0,1,null],[1,2,null],[3,0,null],[2,4,null]],"node_holes":[],"nodes":[{"Action":{"action_name":"common.uuid_generate","label":"GenerateInstanceLockId","name":"saga_instance_lock_id"}},{"Action":{"action_name":"start_instance_update.lock_instance","label":"LockInstance","name":"updater_lock"}},{"Action":{"action_name":"start_instance_update.fetch_state_and_start_real_saga","label":"FetchStateAndStartRealSaga","name":"state"}},{"Start":{"params":{"authz_instance":{"key":"d008f4b2-6063-424c-a333-d54d35c8c058","lookup_type":{"ById":"d008f4b2-6063-424c-a333-d54d35c8c058"},"parent":{"key":"776ccd20-a853-4c93-af22-41ffed980f3e","lookup_type":{"ById":"776ccd20-a853-4c93-af22-41ffed980f3e"},"parent":{"key":"2c668072-a9a1-4a69-928c-e118c99521be","lookup_type":{"ById":"2c668072-a9a1-4a69-928c-e118c99521be"},"parent":null}}},"serialized_authn":{"kind":{"Authenticated":[{"actor":{"UserBuiltin":{"user_builtin_id":"001de000-05e4-4000-8000-000000000002"}}},null]}}}}},"End"]},"saga_name":"start-instance-update","start_node":3} file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/steno-0.4.1/src/sec.rs:1146 saga_id = 0002aa10-b3d2-4ba2-8c02-cc3d7e716b42 saga_name = start-instance-update sec_id = 499bc009-7d23-4916-9be8-0a6a8f4e2f93
1512024-12-14T01:15:18.356ZINFOnexus (SecStore): creating saga file = nexus/db-queries/src/db/sec_store.rs:50 saga_id = 0002aa10-b3d2-4ba2-8c02-cc3d7e716b42 saga_name = start-instance-update
1522024-12-14T01:15:18.357ZINFOnexus (dropshot_internal): received new VMM runtime state from sled agent actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true file = nexus/src/app/instance.rs:2116 local_addr = [fd00:1122:3344:101::c]:12221 method = PUT migration_state = Migrations { migration_in: None, migration_out: None } propolis_id = b2d035ea-f96d-499b-a259-358e56f0523e remote_addr = [fd00:1122:3344:101::1]:50234 req_id = ecfeccfc-1929-46c3-a0cc-d37ecadad30b uri = /vmms/b2d035ea-f96d-499b-a259-358e56f0523e vmm_state = VmmRuntimeState { state: Destroyed, gen: Generation(8), time_updated: 2024-12-14T01:15:18.279588828Z }
1532024-12-14T01:15:18.357ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:50234 req_id = ecfeccfc-1929-46c3-a0cc-d37ecadad30b roles = RoleSet { roles: {} } uri = /vmms/b2d035ea-f96d-499b-a259-358e56f0523e
1542024-12-14T01:15:18.357ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:50234 req_id = ecfeccfc-1929-46c3-a0cc-d37ecadad30b resource = Database result = Ok(()) uri = /vmms/b2d035ea-f96d-499b-a259-358e56f0523e
1552024-12-14T01:15:18.363ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 6515 local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:50234 req_id = ecfeccfc-1929-46c3-a0cc-d37ecadad30b response_code = 204 uri = /vmms/b2d035ea-f96d-499b-a259-358e56f0523e
1562024-12-14T01:15:18.387ZINFOnexus (SagaExecutor): starting saga file = nexus/src/app/saga.rs:353 saga_id = 0002aa10-b3d2-4ba2-8c02-cc3d7e716b42 saga_name = start-instance-update
1572024-12-14T01:15:18.387ZINFOnexus (SEC): saga start file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/steno-0.4.1/src/sec.rs:967 saga_id = 0002aa10-b3d2-4ba2-8c02-cc3d7e716b42 saga_name = start-instance-update sec_id = 499bc009-7d23-4916-9be8-0a6a8f4e2f93
1582024-12-14T01:15:18.387ZDEBGnexus (SEC): saga log event new_state = N003 started sec_id = 499bc009-7d23-4916-9be8-0a6a8f4e2f93
1592024-12-14T01:15:18.387ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 3 saga_id = 0002aa10-b3d2-4ba2-8c02-cc3d7e716b42
1602024-12-14T01:15:18.391ZDEBGnexus (SEC): saga log event new_state = N003 succeeded sec_id = 499bc009-7d23-4916-9be8-0a6a8f4e2f93
1612024-12-14T01:15:18.391ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 3 saga_id = 0002aa10-b3d2-4ba2-8c02-cc3d7e716b42
1622024-12-14T01:15:18.395ZDEBGnexus (SEC): saga log event new_state = N000 started sec_id = 499bc009-7d23-4916-9be8-0a6a8f4e2f93
1632024-12-14T01:15:18.395ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 0 saga_id = 0002aa10-b3d2-4ba2-8c02-cc3d7e716b42
1642024-12-14T01:15:18.398ZDEBGnexus (SEC): saga log event new_state = N000 succeeded sec_id = 499bc009-7d23-4916-9be8-0a6a8f4e2f93
1652024-12-14T01:15:18.398ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(String("6cdd7099-b1d6-454c-9e50-ce1b3fe34715")) node_id = 0 saga_id = 0002aa10-b3d2-4ba2-8c02-cc3d7e716b42
1662024-12-14T01:15:18.401ZDEBGnexus (SEC): saga log event new_state = N001 started sec_id = 499bc009-7d23-4916-9be8-0a6a8f4e2f93
1672024-12-14T01:15:18.401ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 1 saga_id = 0002aa10-b3d2-4ba2-8c02-cc3d7e716b42
1682024-12-14T01:15:18.407ZINFOnexus (SagaExecutor): instance update: attempting to lock instance file = nexus/src/app/sagas/instance_update/start.rs:93 instance_id = d008f4b2-6063-424c-a333-d54d35c8c058 saga_id = 0002aa10-b3d2-4ba2-8c02-cc3d7e716b42 saga_name = start-instance-update
1692024-12-14T01:15:18.407ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 0002aa10-b3d2-4ba2-8c02-cc3d7e716b42 saga_name = start-instance-update saga_node = LockInstance
1702024-12-14T01:15:18.407ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 0002aa10-b3d2-4ba2-8c02-cc3d7e716b42 saga_name = start-instance-update saga_node = LockInstance
1712024-12-14T01:15:18.408ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 0002aa10-b3d2-4ba2-8c02-cc3d7e716b42 saga_name = start-instance-update saga_node = LockInstance
1722024-12-14T01:15:18.409ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 0002aa10-b3d2-4ba2-8c02-cc3d7e716b42 saga_name = start-instance-update saga_node = LockInstance
1732024-12-14T01:15:18.410ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 0002aa10-b3d2-4ba2-8c02-cc3d7e716b42 saga_name = start-instance-update saga_node = LockInstance
1742024-12-14T01:15:18.410ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 0002aa10-b3d2-4ba2-8c02-cc3d7e716b42 saga_name = start-instance-update saga_node = LockInstance
1752024-12-14T01:15:18.412ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 0002aa10-b3d2-4ba2-8c02-cc3d7e716b42 saga_name = start-instance-update saga_node = LockInstance
1762024-12-14T01:15:18.412ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 0002aa10-b3d2-4ba2-8c02-cc3d7e716b42 saga_name = start-instance-update saga_node = LockInstance
1772024-12-14T01:15:18.413ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 0002aa10-b3d2-4ba2-8c02-cc3d7e716b42 saga_name = start-instance-update saga_node = LockInstance
1782024-12-14T01:15:18.413ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 0002aa10-b3d2-4ba2-8c02-cc3d7e716b42 saga_name = start-instance-update saga_node = LockInstance
1792024-12-14T01:15:18.414ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 0002aa10-b3d2-4ba2-8c02-cc3d7e716b42 saga_name = start-instance-update saga_node = LockInstance
1802024-12-14T01:15:18.415ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 0002aa10-b3d2-4ba2-8c02-cc3d7e716b42 saga_name = start-instance-update saga_node = LockInstance
1812024-12-14T01:15:18.416ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } saga_id = 0002aa10-b3d2-4ba2-8c02-cc3d7e716b42 saga_name = start-instance-update saga_node = LockInstance
1822024-12-14T01:15:18.419ZDEBGnexus (SagaExecutor): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Instance { parent: Project { parent: Silo { parent: Fleet, key: 2c668072-a9a1-4a69-928c-e118c99521be, lookup_type: ById(2c668072-a9a1-4a69-928c-e118c99521be) }, key: 776ccd20-a853-4c93-af22-41ffed980f3e, lookup_type: ById(776ccd20-a853-4c93-af22-41ffed980f3e) }, key: d008f4b2-6063-424c-a333-d54d35c8c058, lookup_type: ById(d008f4b2-6063-424c-a333-d54d35c8c058) } result = Ok(()) saga_id = 0002aa10-b3d2-4ba2-8c02-cc3d7e716b42 saga_name = start-instance-update saga_node = LockInstance
1832024-12-14T01:15:18.419ZDEBGnexus (SagaExecutor): attempting to acquire instance updater lock actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true current_gen = Generation(Generation(1)) instance_id = d008f4b2-6063-424c-a333-d54d35c8c058 saga_id = 0002aa10-b3d2-4ba2-8c02-cc3d7e716b42 saga_name = start-instance-update saga_node = LockInstance updater_id = 6cdd7099-b1d6-454c-9e50-ce1b3fe34715
1842024-12-14T01:15:18.419ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 0002aa10-b3d2-4ba2-8c02-cc3d7e716b42 saga_name = start-instance-update saga_node = LockInstance
1852024-12-14T01:15:18.419ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 0002aa10-b3d2-4ba2-8c02-cc3d7e716b42 saga_name = start-instance-update saga_node = LockInstance
1862024-12-14T01:15:18.428ZDEBGnexus (SagaExecutor): attempting to acquire instance updater lock actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true current_gen = Generation(Generation(1)) instance_id = d008f4b2-6063-424c-a333-d54d35c8c058 saga_id = 0002aa10-b3d2-4ba2-8c02-cc3d7e716b42 saga_name = start-instance-update saga_node = LockInstance updater_id = 6cdd7099-b1d6-454c-9e50-ce1b3fe34715
1872024-12-14T01:15:18.428ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 0002aa10-b3d2-4ba2-8c02-cc3d7e716b42 saga_name = start-instance-update saga_node = LockInstance
1882024-12-14T01:15:18.428ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 0002aa10-b3d2-4ba2-8c02-cc3d7e716b42 saga_name = start-instance-update saga_node = LockInstance
1892024-12-14T01:15:18.433ZDEBGnexus (SagaExecutor): instance updater lock acquired! actor_id = 001de000-05e4-4000-8000-000000000002 already_locked = true authenticated = true instance_id = d008f4b2-6063-424c-a333-d54d35c8c058 locked_gen = Generation(Generation(2)) saga_id = 0002aa10-b3d2-4ba2-8c02-cc3d7e716b42 saga_name = start-instance-update saga_node = LockInstance updater_id = 6cdd7099-b1d6-454c-9e50-ce1b3fe34715
1902024-12-14T01:15:18.435ZDEBGnexus (SEC): saga log event new_state = N001 succeeded sec_id = 499bc009-7d23-4916-9be8-0a6a8f4e2f93
1912024-12-14T01:15:18.435ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Object {"locked_gen": Number(2), "updater_id": String("6cdd7099-b1d6-454c-9e50-ce1b3fe34715")}) node_id = 1 saga_id = 0002aa10-b3d2-4ba2-8c02-cc3d7e716b42
1922024-12-14T01:15:18.438ZDEBGnexus (SEC): saga log event new_state = N002 started sec_id = 499bc009-7d23-4916-9be8-0a6a8f4e2f93
1932024-12-14T01:15:18.438ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 2 saga_id = 0002aa10-b3d2-4ba2-8c02-cc3d7e716b42
1942024-12-14T01:15:18.446ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 0002aa10-b3d2-4ba2-8c02-cc3d7e716b42 saga_name = start-instance-update saga_node = FetchStateAndStartRealSaga
1952024-12-14T01:15:18.447ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 0002aa10-b3d2-4ba2-8c02-cc3d7e716b42 saga_name = start-instance-update saga_node = FetchStateAndStartRealSaga
1962024-12-14T01:15:18.448ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 0002aa10-b3d2-4ba2-8c02-cc3d7e716b42 saga_name = start-instance-update saga_node = FetchStateAndStartRealSaga
1972024-12-14T01:15:18.448ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 0002aa10-b3d2-4ba2-8c02-cc3d7e716b42 saga_name = start-instance-update saga_node = FetchStateAndStartRealSaga
1982024-12-14T01:15:18.449ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 0002aa10-b3d2-4ba2-8c02-cc3d7e716b42 saga_name = start-instance-update saga_node = FetchStateAndStartRealSaga
1992024-12-14T01:15:18.449ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 0002aa10-b3d2-4ba2-8c02-cc3d7e716b42 saga_name = start-instance-update saga_node = FetchStateAndStartRealSaga
2002024-12-14T01:15:18.450ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } saga_id = 0002aa10-b3d2-4ba2-8c02-cc3d7e716b42 saga_name = start-instance-update saga_node = FetchStateAndStartRealSaga
2012024-12-14T01:15:18.451ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:57045 req_id = 9ed71953-19df-4db6-9f01-2aab804d366a roles = RoleSet { roles: {} } uri = /metrics/collectors/6e1e4a25-c6a7-49c5-bb1d-86638130bef7/producers?sort_by=id_ascending
2022024-12-14T01:15:18.451ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:57045 req_id = 9ed71953-19df-4db6-9f01-2aab804d366a resource = Database result = Ok(()) uri = /metrics/collectors/6e1e4a25-c6a7-49c5-bb1d-86638130bef7/producers?sort_by=id_ascending
2032024-12-14T01:15:18.453ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1733 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:57045 req_id = 9ed71953-19df-4db6-9f01-2aab804d366a response_code = 200 uri = /metrics/collectors/6e1e4a25-c6a7-49c5-bb1d-86638130bef7/producers?sort_by=id_ascending
2042024-12-14T01:15:18.453ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:57045 req_id = 90b2fda7-2999-431b-8660-37b76c27d66e roles = RoleSet { roles: {} } uri = /metrics/collectors/6e1e4a25-c6a7-49c5-bb1d-86638130bef7/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmNGM2ZGNlZi0yNTc3LTQxMmUtODNkMS0zNzkyZGNjMDQ2MWQifX0%3D
2052024-12-14T01:15:18.453ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:57045 req_id = 90b2fda7-2999-431b-8660-37b76c27d66e resource = Database result = Ok(()) uri = /metrics/collectors/6e1e4a25-c6a7-49c5-bb1d-86638130bef7/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmNGM2ZGNlZi0yNTc3LTQxMmUtODNkMS0zNzkyZGNjMDQ2MWQifX0%3D
2062024-12-14T01:15:18.454ZDEBGnexus (SagaExecutor): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Instance { parent: Project { parent: Silo { parent: Fleet, key: 2c668072-a9a1-4a69-928c-e118c99521be, lookup_type: ById(2c668072-a9a1-4a69-928c-e118c99521be) }, key: 776ccd20-a853-4c93-af22-41ffed980f3e, lookup_type: ById(776ccd20-a853-4c93-af22-41ffed980f3e) }, key: d008f4b2-6063-424c-a333-d54d35c8c058, lookup_type: ById(d008f4b2-6063-424c-a333-d54d35c8c058) } result = Ok(()) saga_id = 0002aa10-b3d2-4ba2-8c02-cc3d7e716b42 saga_name = start-instance-update saga_node = FetchStateAndStartRealSaga
2072024-12-14T01:15:18.454ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 0002aa10-b3d2-4ba2-8c02-cc3d7e716b42 saga_name = start-instance-update saga_node = FetchStateAndStartRealSaga
2082024-12-14T01:15:18.454ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 0002aa10-b3d2-4ba2-8c02-cc3d7e716b42 saga_name = start-instance-update saga_node = FetchStateAndStartRealSaga
2092024-12-14T01:15:18.455ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2036 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:57045 req_id = 90b2fda7-2999-431b-8660-37b76c27d66e response_code = 200 uri = /metrics/collectors/6e1e4a25-c6a7-49c5-bb1d-86638130bef7/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmNGM2ZGNlZi0yNTc3LTQxMmUtODNkMS0zNzkyZGNjMDQ2MWQifX0%3D
2102024-12-14T01:15:18.459ZINFOnexus (SagaExecutor): instance update: starting real update saga... current.active_vmm = Some(Vmm { id: b2d035ea-f96d-499b-a259-358e56f0523e, time_created: 2024-12-14T01:14:57.465339Z, time_deleted: None, instance_id: d008f4b2-6063-424c-a333-d54d35c8c058, sled_id: 7e423e95-464a-471f-a0d9-4799977f0b0f, propolis_ip: V6(Ipv6Network { addr: fd00:1122:3344:101::1:0, prefix: 128 }), propolis_port: SqlU16(12400), runtime: VmmRuntimeState { time_state_updated: 2024-12-14T01:15:18.279588Z, gen: Generation(Generation(8)), state: Destroyed } }) current.migration = None current.runtime_state = InstanceRuntimeState { time_updated: 2024-12-14T01:14:55.295559Z, gen: Generation(Generation(3)), propolis_id: Some(b2d035ea-f96d-499b-a259-358e56f0523e), dst_propolis_id: None, migration_id: None, nexus_state: Vmm, time_last_auto_restarted: None } current.target_vmm = None file = nexus/src/app/sagas/instance_update/start.rs:180 instance_id = d008f4b2-6063-424c-a333-d54d35c8c058 saga_id = 0002aa10-b3d2-4ba2-8c02-cc3d7e716b42 saga_name = start-instance-update update.deprovision = true update.destroy_active_vmm = Some(b2d035ea-f96d-499b-a259-358e56f0523e (propolis)) update.destroy_target_vmm = None update.network_config_update = Some(Delete) update.new_runtime_state = InstanceRuntimeState { time_updated: 2024-12-14T01:15:18.459797943Z, gen: Generation(Generation(4)), propolis_id: None, dst_propolis_id: None, migration_id: None, nexus_state: NoVmm, time_last_auto_restarted: None }
2112024-12-14T01:15:18.464ZINFOnexus (SagaExecutor): preparing saga file = nexus/src/app/saga.rs:255 saga_id = c17231ae-504f-4cf9-a3c8-89ca1429d178 saga_name = instance-update
2122024-12-14T01:15:18.464ZINFOnexus (SEC): saga create dag = {"end_node":14,"graph":{"edge_property":"directed","edges":[[0,1,null],[1,2,null],[2,3,null],[3,4,null],[4,5,null],[5,6,null],[6,7,null],[7,8,null],[9,10,null],[8,9,null],[10,11,null],[11,12,null],[13,0,null],[12,14,null]],"node_holes":[],"nodes":[{"Action":{"action_name":"common.uuid_generate","label":"GenerateInstanceLockId","name":"saga_instance_lock_id"}},{"Action":{"action_name":"instance_update.become_updater","label":"BecomeUpdater","name":"updater_lock"}},{"Constant":{"name":"network_config_update","value":"Delete"}},{"Action":{"action_name":"instance_update.update_network_config","label":"UpdateNetworkConfig","name":"update_network_config"}},{"Action":{"action_name":"instance_update.release_virtual_provisioning","label":"ReleaseVirtualProvisioning","name":"release_virtual_provisioning"}},{"Action":{"action_name":"instance_update.unassign_oximeter_producer","label":"UnassignOximeterProducer","name":"unassign_oximeter_producer"}},{"Action":{"action_name":"instance_update.commit_instance_updates","label":"CommitInstanceUpdates","name":"commit_instance_updates"}},{"Constant":{"name":"destroy_active_vmm_params","value":{"instance_id":"d008f4b2-6063-424c-a333-d54d35c8c058","serialized_authn":{"kind":{"Authenticated":[{"actor":{"UserBuiltin":{"user_builtin_id":"001de000-05e4-4000-8000-000000000002"}}},null]}},"vmm_id":"b2d035ea-f96d-499b-a259-358e56f0523e"}}},{"SubsagaStart":{"params_node_name":"destroy_active_vmm_params","saga_name":"destroy_active_vmm"}},{"Action":{"action_name":"destroy_vmm.release_sled_resources","label":"ReleaseSledResources","name":"release_sled_resources"}},{"Action":{"action_name":"destroy_vmm.mark_vmm_deleted","label":"MarkVmmDeleted","name":"mark_vmm_deleted"}},{"SubsagaEnd":{"name":"active_vmm_destroyed"}},{"Action":{"action_name":"instance_update.chain_successor_saga","label":"ChainSuccessorSaga","name":"chain_successor_saga"}},{"Start":{"params":{"authz_instance":{"key":"d008f4b2-6063-424c-a333-d54d35c8c058","lookup_type":{"ById":"d008f4b2-6063-424c-a333-d54d35c8c058"},"parent":{"key":"776ccd20-a853-4c93-af22-41ffed980f3e","lookup_type":{"ById":"776ccd20-a853-4c93-af22-41ffed980f3e"},"parent":{"key":"2c668072-a9a1-4a69-928c-e118c99521be","lookup_type":{"ById":"2c668072-a9a1-4a69-928c-e118c99521be"},"parent":null}}},"orig_lock":{"locked_gen":2,"updater_id":"6cdd7099-b1d6-454c-9e50-ce1b3fe34715"},"serialized_authn":{"kind":{"Authenticated":[{"actor":{"UserBuiltin":{"user_builtin_id":"001de000-05e4-4000-8000-000000000002"}}},null]}},"update":{"deprovision":{"cpus_diff":2,"project_id":"776ccd20-a853-4c93-af22-41ffed980f3e","ram_diff":1073741824},"destroy_active_vmm":"b2d035ea-f96d-499b-a259-358e56f0523e","destroy_target_vmm":null,"network_config":"Delete","new_runtime":{"dst_propolis_id":null,"gen":4,"migration_id":null,"nexus_state":"NoVmm","propolis_id":null,"time_last_auto_restarted":null,"time_updated":"2024-12-14T01:15:18.459797943Z"}}}}},"End"]},"saga_name":"instance-update","start_node":13} file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/steno-0.4.1/src/sec.rs:1146 saga_id = c17231ae-504f-4cf9-a3c8-89ca1429d178 saga_name = instance-update sec_id = 499bc009-7d23-4916-9be8-0a6a8f4e2f93
2132024-12-14T01:15:18.464ZINFOnexus (SecStore): creating saga file = nexus/db-queries/src/db/sec_store.rs:50 saga_id = c17231ae-504f-4cf9-a3c8-89ca1429d178 saga_name = instance-update
2142024-12-14T01:15:18.500ZINFOnexus (SagaExecutor): starting saga file = nexus/src/app/saga.rs:353 saga_id = c17231ae-504f-4cf9-a3c8-89ca1429d178 saga_name = instance-update
2152024-12-14T01:15:18.500ZINFOnexus (SEC): saga start file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/steno-0.4.1/src/sec.rs:967 saga_id = c17231ae-504f-4cf9-a3c8-89ca1429d178 saga_name = instance-update sec_id = 499bc009-7d23-4916-9be8-0a6a8f4e2f93
2162024-12-14T01:15:18.500ZDEBGnexus (SEC): saga log event new_state = N013 started sec_id = 499bc009-7d23-4916-9be8-0a6a8f4e2f93
2172024-12-14T01:15:18.500ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 13 saga_id = c17231ae-504f-4cf9-a3c8-89ca1429d178
2182024-12-14T01:15:18.503ZDEBGnexus (SEC): saga log event new_state = N013 succeeded sec_id = 499bc009-7d23-4916-9be8-0a6a8f4e2f93
2192024-12-14T01:15:18.503ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 13 saga_id = c17231ae-504f-4cf9-a3c8-89ca1429d178
2202024-12-14T01:15:18.507ZDEBGnexus (SEC): saga log event new_state = N000 started sec_id = 499bc009-7d23-4916-9be8-0a6a8f4e2f93
2212024-12-14T01:15:18.507ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 0 saga_id = c17231ae-504f-4cf9-a3c8-89ca1429d178
2222024-12-14T01:15:18.510ZDEBGnexus (SEC): saga log event new_state = N000 succeeded sec_id = 499bc009-7d23-4916-9be8-0a6a8f4e2f93
2232024-12-14T01:15:18.510ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(String("850bc167-c4ef-4956-af94-afe6433ffd11")) node_id = 0 saga_id = c17231ae-504f-4cf9-a3c8-89ca1429d178
2242024-12-14T01:15:18.514ZDEBGnexus (SEC): saga log event new_state = N001 started sec_id = 499bc009-7d23-4916-9be8-0a6a8f4e2f93
2252024-12-14T01:15:18.514ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 1 saga_id = c17231ae-504f-4cf9-a3c8-89ca1429d178
2262024-12-14T01:15:18.519ZDEBGnexus (SagaExecutor): instance update: trying to become instance updater... instance_id = d008f4b2-6063-424c-a333-d54d35c8c058 parent_lock = UpdaterLock { updater_id: 6cdd7099-b1d6-454c-9e50-ce1b3fe34715, locked_gen: Generation(Generation(2)) } saga_id = c17231ae-504f-4cf9-a3c8-89ca1429d178 saga_name = instance-update
2272024-12-14T01:15:18.519ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = c17231ae-504f-4cf9-a3c8-89ca1429d178 saga_name = instance-update saga_node = BecomeUpdater
2282024-12-14T01:15:18.519ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = c17231ae-504f-4cf9-a3c8-89ca1429d178 saga_name = instance-update saga_node = BecomeUpdater
2292024-12-14T01:15:18.527ZDEBGnexus (SagaExecutor): inherited lock from 6cdd7099-b1d6-454c-9e50-ce1b3fe34715 to 850bc167-c4ef-4956-af94-afe6433ffd11 actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true instance_id = d008f4b2-6063-424c-a333-d54d35c8c058 locked_gen = Generation(Generation(3)) parent_gen = Generation(Generation(2)) parent_id = 6cdd7099-b1d6-454c-9e50-ce1b3fe34715 saga_id = c17231ae-504f-4cf9-a3c8-89ca1429d178 saga_name = instance-update saga_node = BecomeUpdater updater_id = 850bc167-c4ef-4956-af94-afe6433ffd11
2302024-12-14T01:15:18.527ZINFOnexus (SagaExecutor): instance_update: Now, I am become Updater, the destroyer of VMMs. file = nexus/src/app/sagas/instance_update/mod.rs:964 instance_id = d008f4b2-6063-424c-a333-d54d35c8c058 saga_id = c17231ae-504f-4cf9-a3c8-89ca1429d178 saga_name = instance-update
2312024-12-14T01:15:18.527ZDEBGnexus (SEC): saga log event new_state = N001 succeeded sec_id = 499bc009-7d23-4916-9be8-0a6a8f4e2f93
2322024-12-14T01:15:18.527ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Object {"locked_gen": Number(3), "updater_id": String("850bc167-c4ef-4956-af94-afe6433ffd11")}) node_id = 1 saga_id = c17231ae-504f-4cf9-a3c8-89ca1429d178
2332024-12-14T01:15:18.530ZDEBGnexus (SEC): saga log event new_state = N002 started sec_id = 499bc009-7d23-4916-9be8-0a6a8f4e2f93
2342024-12-14T01:15:18.530ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 2 saga_id = c17231ae-504f-4cf9-a3c8-89ca1429d178
2352024-12-14T01:15:18.533ZDEBGnexus (SEC): saga log event new_state = N002 succeeded sec_id = 499bc009-7d23-4916-9be8-0a6a8f4e2f93
2362024-12-14T01:15:18.533ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(String("Delete")) node_id = 2 saga_id = c17231ae-504f-4cf9-a3c8-89ca1429d178
2372024-12-14T01:15:18.536ZDEBGnexus (SEC): saga log event new_state = N003 started sec_id = 499bc009-7d23-4916-9be8-0a6a8f4e2f93
2382024-12-14T01:15:18.536ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 3 saga_id = c17231ae-504f-4cf9-a3c8-89ca1429d178
2392024-12-14T01:15:18.542ZINFOnexus (SagaExecutor): instance update: deleting network config file = nexus/src/app/sagas/instance_update/mod.rs:1017 instance_id = d008f4b2-6063-424c-a333-d54d35c8c058 saga_id = c17231ae-504f-4cf9-a3c8-89ca1429d178 saga_name = instance-update
2402024-12-14T01:15:18.544ZINFOnexus: deleting instance dpd configuration file = nexus/src/app/instance_network.rs:548 instance_id = d008f4b2-6063-424c-a333-d54d35c8c058
2412024-12-14T01:15:18.546ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = c17231ae-504f-4cf9-a3c8-89ca1429d178 saga_name = instance-update saga_node = UpdateNetworkConfig
2422024-12-14T01:15:18.547ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = c17231ae-504f-4cf9-a3c8-89ca1429d178 saga_name = instance-update saga_node = UpdateNetworkConfig
2432024-12-14T01:15:18.550ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = c17231ae-504f-4cf9-a3c8-89ca1429d178 saga_name = instance-update saga_node = UpdateNetworkConfig
2442024-12-14T01:15:18.550ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = c17231ae-504f-4cf9-a3c8-89ca1429d178 saga_name = instance-update saga_node = UpdateNetworkConfig
2452024-12-14T01:15:18.552ZWARNnexus: no matching nat entries to soft delete file = nexus/src/app/instance_network.rs:711
2462024-12-14T01:15:18.553ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = c17231ae-504f-4cf9-a3c8-89ca1429d178 saga_name = instance-update saga_node = UpdateNetworkConfig
2472024-12-14T01:15:18.553ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = c17231ae-504f-4cf9-a3c8-89ca1429d178 saga_name = instance-update saga_node = UpdateNetworkConfig
2482024-12-14T01:15:18.555ZWARNnexus: no matching nat entries to soft delete file = nexus/src/app/instance_network.rs:711
2492024-12-14T01:15:18.555ZDEBGnexus (InstanceAllocator): roles roles = RoleSet { roles: {} }
2502024-12-14T01:15:18.555ZDEBGnexus (InstanceAllocator): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Database result = Ok(())
2512024-12-14T01:15:18.556ZDEBGnexus (InstanceAllocator): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "viewer")} }
2522024-12-14T01:15:18.558ZDEBGnexus (InstanceAllocator): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Fleet result = Ok(())
2532024-12-14T01:15:18.558ZDEBGnexus (InstanceAllocator): roles roles = RoleSet { roles: {} }
2542024-12-14T01:15:18.558ZDEBGnexus (InstanceAllocator): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Database result = Ok(())
2552024-12-14T01:15:18.562ZDEBGnexus: notifying dendrite of updates instance_id = Some(d008f4b2-6063-424c-a333-d54d35c8c058 (instance)) switch = switch0
2562024-12-14T01:15:18.562ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-7e423e95-464a-471f-a0d9-4799977f0b0f.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-7e423e95-464a-471f-a0d9-4799977f0b0f.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1481, tv_nsec: 391353896 } })
2572024-12-14T01:15:18.562ZINFOnexus: Determining switch slots managed by switch zones file = nexus/src/app/mod.rs:1103
2582024-12-14T01:15:18.593ZINFOnexus: determining switch slot managed by dendrite zone file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
2592024-12-14T01:15:18.593ZDEBGnexus (MgsClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
2602024-12-14T01:15:18.593ZDEBGnexus (MgsClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "f3026c9b-a990-4b4b-8c21-5aad76f0adfc", "content-length": "26", "date": "Sat, 14 Dec 2024 01:15:18 GMT"} })
2612024-12-14T01:15:18.593ZINFOnexus: identified switch slot for dendrite zone file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
2622024-12-14T01:15:18.595ZINFOnexus: completed mapping dendrite zones to switch slots file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
2632024-12-14T01:15:18.625ZDEBGnexus (DpdClient): client request body = None method = POST uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/trigger
2642024-12-14T01:15:18.625ZDEBGnexus (DpdClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/trigger", status: 200, headers: {"content-type": "application/json", "x-request-id": "fb70a7a6-6323-47c5-947f-c5c603498db6", "content-length": "4", "date": "Sat, 14 Dec 2024 01:15:17 GMT"} })
2652024-12-14T01:15:18.626ZDEBGnexus (SEC): saga log event new_state = N003 succeeded sec_id = 499bc009-7d23-4916-9be8-0a6a8f4e2f93
2662024-12-14T01:15:18.626ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 3 saga_id = c17231ae-504f-4cf9-a3c8-89ca1429d178
2672024-12-14T01:15:18.630ZDEBGnexus (SEC): saga log event new_state = N004 started sec_id = 499bc009-7d23-4916-9be8-0a6a8f4e2f93
2682024-12-14T01:15:18.630ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 4 saga_id = c17231ae-504f-4cf9-a3c8-89ca1429d178
2692024-12-14T01:15:18.635ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = c17231ae-504f-4cf9-a3c8-89ca1429d178 saga_name = instance-update saga_node = ReleaseVirtualProvisioning
2702024-12-14T01:15:18.635ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = c17231ae-504f-4cf9-a3c8-89ca1429d178 saga_name = instance-update saga_node = ReleaseVirtualProvisioning
2712024-12-14T01:15:18.669ZINFOnexus (SagaExecutor): instance update (no VMM): deallocated virtual provisioning resources file = nexus/src/app/sagas/instance_update/mod.rs:1092 instance_id = d008f4b2-6063-424c-a333-d54d35c8c058 records_deleted = [VirtualProvisioningCollection { id: 001de000-1334-4000-8000-000000000000, time_modified: Some(2024-12-14T01:15:18.638435Z), collection_type: "Fleet", virtual_disk_bytes_provisioned: ByteCount(ByteCount(6442450944)), cpus_provisioned: 0, ram_provisioned: ByteCount(ByteCount(0)) }, VirtualProvisioningCollection { id: 2c668072-a9a1-4a69-928c-e118c99521be, time_modified: Some(2024-12-14T01:15:18.638435Z), collection_type: "Silo", virtual_disk_bytes_provisioned: ByteCount(ByteCount(6442450944)), cpus_provisioned: 0, ram_provisioned: ByteCount(ByteCount(0)) }, VirtualProvisioningCollection { id: 776ccd20-a853-4c93-af22-41ffed980f3e, time_modified: Some(2024-12-14T01:15:18.638435Z), collection_type: "Project", virtual_disk_bytes_provisioned: ByteCount(ByteCount(2147483648)), cpus_provisioned: 0, ram_provisioned: ByteCount(ByteCount(0)) }] saga_id = c17231ae-504f-4cf9-a3c8-89ca1429d178 saga_name = instance-update
2722024-12-14T01:15:18.669ZDEBGnexus (SEC): saga log event new_state = N004 succeeded sec_id = 499bc009-7d23-4916-9be8-0a6a8f4e2f93
2732024-12-14T01:15:18.669ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 4 saga_id = c17231ae-504f-4cf9-a3c8-89ca1429d178
2742024-12-14T01:15:18.673ZDEBGnexus (SEC): saga log event new_state = N005 started sec_id = 499bc009-7d23-4916-9be8-0a6a8f4e2f93
2752024-12-14T01:15:18.673ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 5 saga_id = c17231ae-504f-4cf9-a3c8-89ca1429d178
2762024-12-14T01:15:18.674ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 1eb4b98e-b257-4f18-868f-af0945b7c1ce roles = RoleSet { roles: {} } uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
2772024-12-14T01:15:18.674ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 1eb4b98e-b257-4f18-868f-af0945b7c1ce resource = Database result = Ok(()) uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
2782024-12-14T01:15:18.676ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2127 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 1eb4b98e-b257-4f18-868f-af0945b7c1ce response_code = 200 uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
2792024-12-14T01:15:18.677ZINFOnexus (SagaExecutor): instance update (no VMM): unassigning oximeter producer file = nexus/src/app/sagas/instance_update/mod.rs:1127 instance_id = d008f4b2-6063-424c-a333-d54d35c8c058 saga_id = c17231ae-504f-4cf9-a3c8-89ca1429d178 saga_name = instance-update
2802024-12-14T01:15:18.679ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = c17231ae-504f-4cf9-a3c8-89ca1429d178 saga_name = instance-update saga_node = UnassignOximeterProducer
2812024-12-14T01:15:18.679ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = c17231ae-504f-4cf9-a3c8-89ca1429d178 saga_name = instance-update saga_node = UnassignOximeterProducer
2822024-12-14T01:15:18.683ZDEBGnexus (SagaExecutor): deleted metric producer assignment collector_id = 6e1e4a25-c6a7-49c5-bb1d-86638130bef7 producer_id = d008f4b2-6063-424c-a333-d54d35c8c058 saga_id = c17231ae-504f-4cf9-a3c8-89ca1429d178 saga_name = instance-update
2832024-12-14T01:15:18.683ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = c17231ae-504f-4cf9-a3c8-89ca1429d178 saga_name = instance-update saga_node = UnassignOximeterProducer
2842024-12-14T01:15:18.683ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = c17231ae-504f-4cf9-a3c8-89ca1429d178 saga_name = instance-update saga_node = UnassignOximeterProducer
2852024-12-14T01:15:18.719ZINFOnexus (SagaExecutor): registered oximeter collector client file = nexus/src/app/oximeter.rs:262 id = d008f4b2-6063-424c-a333-d54d35c8c058 saga_id = c17231ae-504f-4cf9-a3c8-89ca1429d178 saga_name = instance-update
2862024-12-14T01:15:18.721ZDEBGnexus (SagaExecutor): client request body = None method = DELETE oximeter-collector = d008f4b2-6063-424c-a333-d54d35c8c058 saga_id = c17231ae-504f-4cf9-a3c8-89ca1429d178 saga_name = instance-update uri = http://[fd00:1122:3344:101::d]:12223/producers/d008f4b2-6063-424c-a333-d54d35c8c058
2872024-12-14T01:15:18.736ZDEBGnexus (SagaExecutor): client response oximeter-collector = d008f4b2-6063-424c-a333-d54d35c8c058 result = Ok(Response { url: "http://[fd00:1122:3344:101::d]:12223/producers/d008f4b2-6063-424c-a333-d54d35c8c058", status: 204, headers: {"x-request-id": "1606286b-453b-45a9-82c0-0fa97a9b136b", "date": "Sat, 14 Dec 2024 01:15:18 GMT"} }) saga_id = c17231ae-504f-4cf9-a3c8-89ca1429d178 saga_name = instance-update
2882024-12-14T01:15:18.736ZDEBGnexus (SagaExecutor): successfully deleted producer from collector address = [fd00:1122:3344:101::d]:12223 collector_id = 6e1e4a25-c6a7-49c5-bb1d-86638130bef7 producer_id = d008f4b2-6063-424c-a333-d54d35c8c058 saga_id = c17231ae-504f-4cf9-a3c8-89ca1429d178 saga_name = instance-update
2892024-12-14T01:15:18.737ZDEBGnexus (SEC): saga log event new_state = N005 succeeded sec_id = 499bc009-7d23-4916-9be8-0a6a8f4e2f93
2902024-12-14T01:15:18.737ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 5 saga_id = c17231ae-504f-4cf9-a3c8-89ca1429d178
2912024-12-14T01:15:18.741ZDEBGnexus (SEC): saga log event new_state = N006 started sec_id = 499bc009-7d23-4916-9be8-0a6a8f4e2f93
2922024-12-14T01:15:18.741ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 6 saga_id = c17231ae-504f-4cf9-a3c8-89ca1429d178
2932024-12-14T01:15:18.744ZDEBGnexus (SagaExecutor): instance update: committing new runtime state and unlocking... instance_id = d008f4b2-6063-424c-a333-d54d35c8c058 lock = UpdaterLock { updater_id: 850bc167-c4ef-4956-af94-afe6433ffd11, locked_gen: Generation(Generation(3)) } new_runtime = InstanceRuntimeState { time_updated: 2024-12-14T01:15:18.459797943Z, gen: Generation(Generation(4)), propolis_id: None, dst_propolis_id: None, migration_id: None, nexus_state: NoVmm, time_last_auto_restarted: None } saga_id = c17231ae-504f-4cf9-a3c8-89ca1429d178 saga_name = instance-update
2942024-12-14T01:15:18.744ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = c17231ae-504f-4cf9-a3c8-89ca1429d178 saga_name = instance-update saga_node = CommitInstanceUpdates
2952024-12-14T01:15:18.744ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = c17231ae-504f-4cf9-a3c8-89ca1429d178 saga_name = instance-update saga_node = CommitInstanceUpdates
2962024-12-14T01:15:18.780ZINFOnexus (SagaExecutor): instance update: committed update new runtime state! did_unlock = true file = nexus/src/app/sagas/instance_update/mod.rs:1176 instance_id = d008f4b2-6063-424c-a333-d54d35c8c058 new_runtime = InstanceRuntimeState { time_updated: 2024-12-14T01:15:18.459797943Z, gen: Generation(Generation(4)), propolis_id: None, dst_propolis_id: None, migration_id: None, nexus_state: NoVmm, time_last_auto_restarted: None } saga_id = c17231ae-504f-4cf9-a3c8-89ca1429d178 saga_name = instance-update
2972024-12-14T01:15:18.780ZDEBGnexus (SEC): saga log event new_state = N006 succeeded sec_id = 499bc009-7d23-4916-9be8-0a6a8f4e2f93
2982024-12-14T01:15:18.780ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 6 saga_id = c17231ae-504f-4cf9-a3c8-89ca1429d178
2992024-12-14T01:15:18.780ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 25 reason = Signaled
3002024-12-14T01:15:18.780ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
3012024-12-14T01:15:18.780ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 26 reason = Signaled
3022024-12-14T01:15:18.780ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
3032024-12-14T01:15:18.780ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
3042024-12-14T01:15:18.780ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
3052024-12-14T01:15:18.780ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
3062024-12-14T01:15:18.782ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3072024-12-14T01:15:18.783ZDEBGnexus (SEC): saga log event new_state = N007 started sec_id = 499bc009-7d23-4916-9be8-0a6a8f4e2f93
3082024-12-14T01:15:18.783ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 7 saga_id = c17231ae-504f-4cf9-a3c8-89ca1429d178
3092024-12-14T01:15:18.784ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
3102024-12-14T01:15:18.784ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
3112024-12-14T01:15:18.784ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
3122024-12-14T01:15:18.785ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3132024-12-14T01:15:18.786ZDEBGnexus (SEC): saga log event new_state = N007 succeeded sec_id = 499bc009-7d23-4916-9be8-0a6a8f4e2f93
3142024-12-14T01:15:18.786ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Object {"instance_id": String("d008f4b2-6063-424c-a333-d54d35c8c058"), "serialized_authn": Object {"kind": Object {"Authenticated": Array [Object {"actor": Object {"UserBuiltin": Object {"user_builtin_id": String("001de000-05e4-4000-8000-000000000002")}}}, Null]}}, "vmm_id": String("b2d035ea-f96d-499b-a259-358e56f0523e")}) node_id = 7 saga_id = c17231ae-504f-4cf9-a3c8-89ca1429d178
3152024-12-14T01:15:18.787ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
3162024-12-14T01:15:18.787ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
3172024-12-14T01:15:18.787ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
3182024-12-14T01:15:18.788ZDEBGnexus (SEC): saga log event new_state = N008 started sec_id = 499bc009-7d23-4916-9be8-0a6a8f4e2f93
3192024-12-14T01:15:18.788ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 8 saga_id = c17231ae-504f-4cf9-a3c8-89ca1429d178
3202024-12-14T01:15:18.791ZDEBGnexus (SEC): saga log event new_state = N008 succeeded sec_id = 499bc009-7d23-4916-9be8-0a6a8f4e2f93
3212024-12-14T01:15:18.792ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 8 saga_id = c17231ae-504f-4cf9-a3c8-89ca1429d178
3222024-12-14T01:15:18.795ZDEBGnexus (SEC): saga log event new_state = N009 started sec_id = 499bc009-7d23-4916-9be8-0a6a8f4e2f93
3232024-12-14T01:15:18.795ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 9 saga_id = c17231ae-504f-4cf9-a3c8-89ca1429d178
3242024-12-14T01:15:18.798ZINFOnexus (SagaExecutor): instance update (VMM destroyed): deallocating sled resource reservation file = nexus/src/app/sagas/instance_update/destroyed.rs:83 instance_id = d008f4b2-6063-424c-a333-d54d35c8c058 propolis_id = b2d035ea-f96d-499b-a259-358e56f0523e saga_id = c17231ae-504f-4cf9-a3c8-89ca1429d178 saga_name = instance-update
3252024-12-14T01:15:18.798ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = c17231ae-504f-4cf9-a3c8-89ca1429d178 saga_name = instance-update saga_node = ReleaseSledResources
3262024-12-14T01:15:18.798ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = c17231ae-504f-4cf9-a3c8-89ca1429d178 saga_name = instance-update saga_node = ReleaseSledResources
3272024-12-14T01:15:18.808ZDEBGnexus (SEC): saga log event new_state = N009 succeeded sec_id = 499bc009-7d23-4916-9be8-0a6a8f4e2f93
3282024-12-14T01:15:18.808ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 9 saga_id = c17231ae-504f-4cf9-a3c8-89ca1429d178
3292024-12-14T01:15:18.820ZINFOnexus (BackgroundTasks): VPC route manager sled 7e423e95-464a-471f-a0d9-4799977f0b0f background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
3302024-12-14T01:15:18.820ZDEBGnexus (BackgroundTasks): client request SledAgent = 7e423e95-464a-471f-a0d9-4799977f0b0f background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
3312024-12-14T01:15:18.820ZDEBGnexus (BackgroundTasks): client response SledAgent = 7e423e95-464a-471f-a0d9-4799977f0b0f background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "08257fb5-6276-44d0-9db1-6705c62e1b4c", "content-length": "372", "date": "Sat, 14 Dec 2024 01:15:18 GMT"} })
3322024-12-14T01:15:18.820ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
3332024-12-14T01:15:18.820ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
3342024-12-14T01:15:18.823ZDEBGnexus (SEC): saga log event new_state = N010 started sec_id = 499bc009-7d23-4916-9be8-0a6a8f4e2f93
3352024-12-14T01:15:18.823ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 10 saga_id = c17231ae-504f-4cf9-a3c8-89ca1429d178
3362024-12-14T01:15:18.826ZINFOnexus (SagaExecutor): instance update (VMM destroyed): marking VMM record deleted file = nexus/src/app/sagas/instance_update/destroyed.rs:114 instance_id = d008f4b2-6063-424c-a333-d54d35c8c058 propolis_id = b2d035ea-f96d-499b-a259-358e56f0523e saga_id = c17231ae-504f-4cf9-a3c8-89ca1429d178 saga_name = instance-update
3372024-12-14T01:15:18.826ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = c17231ae-504f-4cf9-a3c8-89ca1429d178 saga_name = instance-update saga_node = MarkVmmDeleted
3382024-12-14T01:15:18.827ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = c17231ae-504f-4cf9-a3c8-89ca1429d178 saga_name = instance-update saga_node = MarkVmmDeleted
3392024-12-14T01:15:18.845ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
3402024-12-14T01:15:18.845ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
3412024-12-14T01:15:18.865ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {bdac77ef-64e7-4acf-b95d-9c4d3e7809db: {10.151.1.175: {18932050-af06-4b0f-bd31-d6d657d6e3ef}, 10.151.1.176: {18932050-af06-4b0f-bd31-d6d657d6e3ef}}} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = buskin
3422024-12-14T01:15:18.865ZDEBGnexus (BackgroundTasks): client request SledAgent = 7e423e95-464a-471f-a0d9-4799977f0b0f background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
3432024-12-14T01:15:18.865ZDEBGnexus (BackgroundTasks): client response SledAgent = 7e423e95-464a-471f-a0d9-4799977f0b0f background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "f9f97094-8861-4714-901a-eeab0e1b9a6b", "date": "Sat, 14 Dec 2024 01:15:18 GMT"} })
3442024-12-14T01:15:18.865ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
3452024-12-14T01:15:18.865ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
3462024-12-14T01:15:18.868ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
3472024-12-14T01:15:18.868ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
3482024-12-14T01:15:18.869ZDEBGnexus (SEC): saga log event new_state = N010 succeeded sec_id = 499bc009-7d23-4916-9be8-0a6a8f4e2f93
3492024-12-14T01:15:18.869ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 10 saga_id = c17231ae-504f-4cf9-a3c8-89ca1429d178
3502024-12-14T01:15:18.871ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
3512024-12-14T01:15:18.871ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
3522024-12-14T01:15:18.872ZDEBGnexus (SEC): saga log event new_state = N011 started sec_id = 499bc009-7d23-4916-9be8-0a6a8f4e2f93
3532024-12-14T01:15:18.872ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 11 saga_id = c17231ae-504f-4cf9-a3c8-89ca1429d178
3542024-12-14T01:15:18.874ZINFOnexus (BackgroundTasks): VPC route manager sled 7e423e95-464a-471f-a0d9-4799977f0b0f push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
3552024-12-14T01:15:18.874ZDEBGnexus (BackgroundTasks): client request SledAgent = 7e423e95-464a-471f-a0d9-4799977f0b0f background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
3562024-12-14T01:15:18.874ZDEBGnexus (BackgroundTasks): client response SledAgent = 7e423e95-464a-471f-a0d9-4799977f0b0f background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "c2084733-14a3-4b4f-b6e7-e75ba08987aa", "date": "Sat, 14 Dec 2024 01:15:18 GMT"} })
3572024-12-14T01:15:18.876ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 96.113662ms iteration = 26 status = {}
3582024-12-14T01:15:18.876ZDEBGnexus (SEC): saga log event new_state = N011 succeeded sec_id = 499bc009-7d23-4916-9be8-0a6a8f4e2f93
3592024-12-14T01:15:18.876ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 11 saga_id = c17231ae-504f-4cf9-a3c8-89ca1429d178
3602024-12-14T01:15:18.879ZDEBGnexus (SEC): saga log event new_state = N012 started sec_id = 499bc009-7d23-4916-9be8-0a6a8f4e2f93
3612024-12-14T01:15:18.879ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 12 saga_id = c17231ae-504f-4cf9-a3c8-89ca1429d178
3622024-12-14T01:15:18.883ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = c17231ae-504f-4cf9-a3c8-89ca1429d178 saga_name = instance-update saga_node = ChainSuccessorSaga
3632024-12-14T01:15:18.883ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = c17231ae-504f-4cf9-a3c8-89ca1429d178 saga_name = instance-update saga_node = ChainSuccessorSaga
3642024-12-14T01:15:18.884ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = c17231ae-504f-4cf9-a3c8-89ca1429d178 saga_name = instance-update saga_node = ChainSuccessorSaga
3652024-12-14T01:15:18.884ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = c17231ae-504f-4cf9-a3c8-89ca1429d178 saga_name = instance-update saga_node = ChainSuccessorSaga
3662024-12-14T01:15:18.885ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = c17231ae-504f-4cf9-a3c8-89ca1429d178 saga_name = instance-update saga_node = ChainSuccessorSaga
3672024-12-14T01:15:18.886ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = c17231ae-504f-4cf9-a3c8-89ca1429d178 saga_name = instance-update saga_node = ChainSuccessorSaga
3682024-12-14T01:15:18.887ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } saga_id = c17231ae-504f-4cf9-a3c8-89ca1429d178 saga_name = instance-update saga_node = ChainSuccessorSaga
3692024-12-14T01:15:18.890ZDEBGnexus (SagaExecutor): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Instance { parent: Project { parent: Silo { parent: Fleet, key: 2c668072-a9a1-4a69-928c-e118c99521be, lookup_type: ById(2c668072-a9a1-4a69-928c-e118c99521be) }, key: 776ccd20-a853-4c93-af22-41ffed980f3e, lookup_type: ById(776ccd20-a853-4c93-af22-41ffed980f3e) }, key: d008f4b2-6063-424c-a333-d54d35c8c058, lookup_type: ById(d008f4b2-6063-424c-a333-d54d35c8c058) } result = Ok(()) saga_id = c17231ae-504f-4cf9-a3c8-89ca1429d178 saga_name = instance-update saga_node = ChainSuccessorSaga
3702024-12-14T01:15:18.890ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = c17231ae-504f-4cf9-a3c8-89ca1429d178 saga_name = instance-update saga_node = ChainSuccessorSaga
3712024-12-14T01:15:18.890ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = c17231ae-504f-4cf9-a3c8-89ca1429d178 saga_name = instance-update saga_node = ChainSuccessorSaga
3722024-12-14T01:15:18.893ZDEBGnexus (SagaExecutor): instance update: instance will not reincarnate auto_restart_config = InstanceAutoRestart { policy: None, cooldown: None } instance_id = d008f4b2-6063-424c-a333-d54d35c8c058 karmic_state = WillReincarnate needs_reincarnation = false saga_id = c17231ae-504f-4cf9-a3c8-89ca1429d178 saga_name = instance-update
3732024-12-14T01:15:18.893ZDEBGnexus (SEC): saga log event new_state = N012 succeeded sec_id = 499bc009-7d23-4916-9be8-0a6a8f4e2f93
3742024-12-14T01:15:18.893ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 12 saga_id = c17231ae-504f-4cf9-a3c8-89ca1429d178
3752024-12-14T01:15:18.897ZDEBGnexus (SEC): saga log event new_state = N014 started sec_id = 499bc009-7d23-4916-9be8-0a6a8f4e2f93
3762024-12-14T01:15:18.897ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 14 saga_id = c17231ae-504f-4cf9-a3c8-89ca1429d178
3772024-12-14T01:15:18.901ZDEBGnexus (SEC): saga log event new_state = N014 succeeded sec_id = 499bc009-7d23-4916-9be8-0a6a8f4e2f93
3782024-12-14T01:15:18.901ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 14 saga_id = c17231ae-504f-4cf9-a3c8-89ca1429d178
3792024-12-14T01:15:18.904ZINFOnexus (SEC): update for saga cached state file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/steno-0.4.1/src/sec.rs:1384 new_state = Done saga_id = c17231ae-504f-4cf9-a3c8-89ca1429d178 sec_id = 499bc009-7d23-4916-9be8-0a6a8f4e2f93
3802024-12-14T01:15:18.904ZINFOnexus (SecStore): updating state file = nexus/db-queries/src/db/sec_store.rs:95 new_state = done saga_id = c17231ae-504f-4cf9-a3c8-89ca1429d178
3812024-12-14T01:15:18.942ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
3822024-12-14T01:15:18.942ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
3832024-12-14T01:15:18.947ZINFOnexus (SEC): saga finished file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/steno-0.4.1/src/sec.rs:1025 result = success saga_id = c17231ae-504f-4cf9-a3c8-89ca1429d178 saga_name = instance-update sec_id = 499bc009-7d23-4916-9be8-0a6a8f4e2f93
3842024-12-14T01:15:18.947ZINFOnexus (SagaExecutor): saga finished file = nexus/src/app/saga.rs:404 saga_id = c17231ae-504f-4cf9-a3c8-89ca1429d178 saga_name = instance-update saga_result = SagaResult { saga_id: c17231ae-504f-4cf9-a3c8-89ca1429d178, saga_log: SagaLog { saga_id: c17231ae-504f-4cf9-a3c8-89ca1429d178, unwinding: false, events: [N013 started, N013 succeeded, N000 started, N000 succeeded, N001 started, N001 succeeded, N002 started, N002 succeeded, N003 started, N003 succeeded, N004 started, N004 succeeded, N005 started, N005 succeeded, N006 started, N006 succeeded, N007 started, N007 succeeded, N008 started, N008 succeeded, N009 started, N009 succeeded, N010 started, N010 succeeded, N011 started, N011 succeeded, N012 started, N012 succeeded, N014 started, N014 succeeded], node_status: {0: Succeeded(String("850bc167-c4ef-4956-af94-afe6433ffd11")), 1: Succeeded(Object {"locked_gen": Number(3), "updater_id": String("850bc167-c4ef-4956-af94-afe6433ffd11")}), 2: Succeeded(String("Delete")), 3: Succeeded(Null), 4: Succeeded(Null), 5: Succeeded(Null), 6: Succeeded(Null), 7: Succeeded(Object {"instance_id": String("d008f4b2-6063-424c-a333-d54d35c8c058"), "serialized_authn": Object {"kind": Object {"Authenticated": Array [Object {"actor": Object {"UserBuiltin": Object {"user_builtin_id": String("001de000-05e4-4000-8000-000000000002")}}}, Null]}}, "vmm_id": String("b2d035ea-f96d-499b-a259-358e56f0523e")}), 8: Succeeded(Null), 9: Succeeded(Null), 10: Succeeded(Null), 11: Succeeded(Null), 12: Succeeded(Null), 13: Succeeded(Null), 14: Succeeded(Null)} }, kind: Ok(SagaResultOk { saga_output: Null, node_outputs: {"active_vmm_destroyed": Null, "chain_successor_saga": Null, "commit_instance_updates": Null, "destroy_active_vmm_params": Object {"instance_id": String("d008f4b2-6063-424c-a333-d54d35c8c058"), "serialized_authn": Object {"kind": Object {"Authenticated": Array [Object {"actor": Object {"UserBuiltin": Object {"user_builtin_id": String("001de000-05e4-4000-8000-000000000002")}}}, Null]}}, "vmm_id": String("b2d035ea-f96d-499b-a259-358e56f0523e")}, "mark_vmm_deleted": Null, "network_config_update": String("Delete"), "release_sled_resources": Null, "release_virtual_provisioning": Null, "saga_instance_lock_id": String("850bc167-c4ef-4956-af94-afe6433ffd11"), "unassign_oximeter_producer": Null, "update_network_config": Null, "updater_lock": Object {"locked_gen": Number(3), "updater_id": String("850bc167-c4ef-4956-af94-afe6433ffd11")}} }) }
3852024-12-14T01:15:18.947ZDEBGnexus (SagaExecutor): instance update: child saga completed successfully child_saga_id = c17231ae-504f-4cf9-a3c8-89ca1429d178 instance_id = d008f4b2-6063-424c-a333-d54d35c8c058 saga_id = 0002aa10-b3d2-4ba2-8c02-cc3d7e716b42 saga_name = start-instance-update
3862024-12-14T01:15:18.947ZDEBGnexus (SEC): saga log event new_state = N002 succeeded sec_id = 499bc009-7d23-4916-9be8-0a6a8f4e2f93
3872024-12-14T01:15:18.947ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 2 saga_id = 0002aa10-b3d2-4ba2-8c02-cc3d7e716b42
3882024-12-14T01:15:18.952ZDEBGnexus (SEC): saga log event new_state = N004 started sec_id = 499bc009-7d23-4916-9be8-0a6a8f4e2f93
3892024-12-14T01:15:18.952ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 4 saga_id = 0002aa10-b3d2-4ba2-8c02-cc3d7e716b42
3902024-12-14T01:15:18.955ZDEBGnexus (SEC): saga log event new_state = N004 succeeded sec_id = 499bc009-7d23-4916-9be8-0a6a8f4e2f93
3912024-12-14T01:15:18.955ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 4 saga_id = 0002aa10-b3d2-4ba2-8c02-cc3d7e716b42
3922024-12-14T01:15:18.958ZINFOnexus (SEC): update for saga cached state file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/steno-0.4.1/src/sec.rs:1384 new_state = Done saga_id = 0002aa10-b3d2-4ba2-8c02-cc3d7e716b42 sec_id = 499bc009-7d23-4916-9be8-0a6a8f4e2f93
3932024-12-14T01:15:18.958ZINFOnexus (SecStore): updating state file = nexus/db-queries/src/db/sec_store.rs:95 new_state = done saga_id = 0002aa10-b3d2-4ba2-8c02-cc3d7e716b42
3942024-12-14T01:15:18.964ZINFOnexus (SEC): saga finished file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/steno-0.4.1/src/sec.rs:1025 result = success saga_id = 0002aa10-b3d2-4ba2-8c02-cc3d7e716b42 saga_name = start-instance-update sec_id = 499bc009-7d23-4916-9be8-0a6a8f4e2f93
3952024-12-14T01:15:18.964ZINFOnexus (SagaExecutor): saga finished file = nexus/src/app/saga.rs:404 saga_id = 0002aa10-b3d2-4ba2-8c02-cc3d7e716b42 saga_name = start-instance-update saga_result = SagaResult { saga_id: 0002aa10-b3d2-4ba2-8c02-cc3d7e716b42, saga_log: SagaLog { saga_id: 0002aa10-b3d2-4ba2-8c02-cc3d7e716b42, unwinding: false, events: [N003 started, N003 succeeded, N000 started, N000 succeeded, N001 started, N001 succeeded, N002 started, N002 succeeded, N004 started, N004 succeeded], node_status: {0: Succeeded(String("6cdd7099-b1d6-454c-9e50-ce1b3fe34715")), 1: Succeeded(Object {"locked_gen": Number(2), "updater_id": String("6cdd7099-b1d6-454c-9e50-ce1b3fe34715")}), 2: Succeeded(Null), 3: Succeeded(Null), 4: Succeeded(Null)} }, kind: Ok(SagaResultOk { saga_output: Null, node_outputs: {"saga_instance_lock_id": String("6cdd7099-b1d6-454c-9e50-ce1b3fe34715"), "state": Null, "updater_lock": Object {"locked_gen": Number(2), "updater_id": String("6cdd7099-b1d6-454c-9e50-ce1b3fe34715")}} }) }
3962024-12-14T01:15:18.969ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3972024-12-14T01:15:18.971ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
3982024-12-14T01:15:18.971ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
3992024-12-14T01:15:18.972ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
4002024-12-14T01:15:18.999ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4012024-12-14T01:15:19.001ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
4022024-12-14T01:15:19.001ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
4032024-12-14T01:15:19.001ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
4042024-12-14T01:15:19.074ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:43741 req_id = 2ad3e3bf-7fc3-49ae-b530-0bdc1d504a33 roles = RoleSet { roles: {} } uri = /nat/ipv4/changeset/10?limit=100
4052024-12-14T01:15:19.074ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:43741 req_id = 2ad3e3bf-7fc3-49ae-b530-0bdc1d504a33 resource = Database result = Ok(()) uri = /nat/ipv4/changeset/10?limit=100
4062024-12-14T01:15:19.077ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 3247 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:43741 req_id = 2ad3e3bf-7fc3-49ae-b530-0bdc1d504a33 response_code = 200 uri = /nat/ipv4/changeset/10?limit=100
4072024-12-14T01:15:19.087ZDEBGnexus (BackgroundTasks): client request SledAgent = 7e423e95-464a-471f-a0d9-4799977f0b0f background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
4082024-12-14T01:15:19.087ZDEBGnexus (BackgroundTasks): client response SledAgent = 7e423e95-464a-471f-a0d9-4799977f0b0f background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "ed210454-1db1-4999-b7ff-bc2abeaffc13", "content-length": "691", "date": "Sat, 14 Dec 2024 01:15:18 GMT"} })
4092024-12-14T01:15:19.087ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = buskin
4102024-12-14T01:15:19.087ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = buskin
4112024-12-14T01:15:19.087ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = buskin
4122024-12-14T01:15:19.089ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 309.288581ms iteration = 25 status = {}
4132024-12-14T01:15:19.697ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = afaa5075-3385-4a93-b467-717005d1c05a roles = RoleSet { roles: {} } uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
4142024-12-14T01:15:19.697ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = afaa5075-3385-4a93-b467-717005d1c05a resource = Database result = Ok(()) uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
4152024-12-14T01:15:19.699ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2542 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = afaa5075-3385-4a93-b467-717005d1c05a response_code = 200 uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
4162024-12-14T01:15:20.079ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:43741 req_id = 1ce1462a-4a5b-47df-b25d-0b7f5b8e9f9b roles = RoleSet { roles: {} } uri = /nat/ipv4/changeset/10?limit=100
4172024-12-14T01:15:20.079ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:43741 req_id = 1ce1462a-4a5b-47df-b25d-0b7f5b8e9f9b resource = Database result = Ok(()) uri = /nat/ipv4/changeset/10?limit=100
4182024-12-14T01:15:20.082ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 3159 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:43741 req_id = 1ce1462a-4a5b-47df-b25d-0b7f5b8e9f9b response_code = 200 uri = /nat/ipv4/changeset/10?limit=100
4192024-12-14T01:15:20.719ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = dc84e8fa-2abf-4f4a-8090-ca08a0200da2 roles = RoleSet { roles: {} } uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
4202024-12-14T01:15:20.719ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = dc84e8fa-2abf-4f4a-8090-ca08a0200da2 resource = Database result = Ok(()) uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
4212024-12-14T01:15:20.721ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2140 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = dc84e8fa-2abf-4f4a-8090-ca08a0200da2 response_code = 200 uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
4222024-12-14T01:15:21.742ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = e22dab55-0125-4dd4-8234-f72a36d798a0 roles = RoleSet { roles: {} } uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
4232024-12-14T01:15:21.742ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = e22dab55-0125-4dd4-8234-f72a36d798a0 resource = Database result = Ok(()) uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
4242024-12-14T01:15:21.745ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2862 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = e22dab55-0125-4dd4-8234-f72a36d798a0 response_code = 200 uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
4252024-12-14T01:15:22.765ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 7eaa65a3-7ec5-4128-b7bd-2a1f96c6888b roles = RoleSet { roles: {} } uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
4262024-12-14T01:15:22.765ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 7eaa65a3-7ec5-4128-b7bd-2a1f96c6888b resource = Database result = Ok(()) uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
4272024-12-14T01:15:22.767ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2654 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 7eaa65a3-7ec5-4128-b7bd-2a1f96c6888b response_code = 200 uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
4282024-12-14T01:15:23.462ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1594 local_addr = [fd00:1122:3344:101::c]:54941 method = GET remote_addr = [fd00:1122:3344:101::d]:43526 req_id = 1515b26a-87f5-4750-93d6-5bddf338b98b response_code = 200 uri = /499bc009-7d23-4916-9be8-0a6a8f4e2f93
4292024-12-14T01:15:23.787ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 3f8c51f2-6dba-4136-b553-4d9fcf1ce007 roles = RoleSet { roles: {} } uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
4302024-12-14T01:15:23.788ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 3f8c51f2-6dba-4136-b553-4d9fcf1ce007 resource = Database result = Ok(()) uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
4312024-12-14T01:15:23.790ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2522 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 3f8c51f2-6dba-4136-b553-4d9fcf1ce007 response_code = 200 uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
4322024-12-14T01:15:24.747ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 13 reason = Timeout
4332024-12-14T01:15:24.747ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:56
4342024-12-14T01:15:24.747ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 25 reason = Timeout
4352024-12-14T01:15:24.747ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
4362024-12-14T01:15:24.747ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
4372024-12-14T01:15:24.747ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 25 reason = Timeout
4382024-12-14T01:15:24.747ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
4392024-12-14T01:15:24.747ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
4402024-12-14T01:15:24.747ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 25 reason = Timeout
4412024-12-14T01:15:24.747ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 25 reason = Timeout
4422024-12-14T01:15:24.747ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
4432024-12-14T01:15:24.747ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
4442024-12-14T01:15:24.747ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 13 reason = Timeout
4452024-12-14T01:15:24.747ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
4462024-12-14T01:15:24.747ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 25 reason = Timeout
4472024-12-14T01:15:24.747ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
4482024-12-14T01:15:24.747ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 14 reason = Timeout
4492024-12-14T01:15:24.747ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:83 reason = blueprint disabled target_id = 8770bfcc-3a71-4738-a6b0-91008cadcb0f
4502024-12-14T01:15:24.747ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 16.1\u{b5}s iteration = 14 status = {"enabled":false,"target_id":"8770bfcc-3a71-4738-a6b0-91008cadcb0f"}
4512024-12-14T01:15:24.747ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 27 reason = Timeout
4522024-12-14T01:15:24.747ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
4532024-12-14T01:15:24.747ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
4542024-12-14T01:15:24.747ZDEBGnexus (BackgroundTasks): activating background_task = crdb_node_id_collector iteration = 6 reason = Timeout
4552024-12-14T01:15:24.747ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 25 reason = Timeout
4562024-12-14T01:15:24.747ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
4572024-12-14T01:15:24.747ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 14 reason = Timeout
4582024-12-14T01:15:24.747ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
4592024-12-14T01:15:24.747ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
4602024-12-14T01:15:24.747ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
4612024-12-14T01:15:24.747ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 73 reason = Timeout
4622024-12-14T01:15:24.747ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 25 reason = Timeout
4632024-12-14T01:15:24.747ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4642024-12-14T01:15:24.747ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
4652024-12-14T01:15:24.747ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 25 reason = Timeout
4662024-12-14T01:15:24.747ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
4672024-12-14T01:15:24.747ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 15 reason = Timeout
4682024-12-14T01:15:24.747ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 15 reason = Timeout
4692024-12-14T01:15:24.747ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 13 reason = Timeout
4702024-12-14T01:15:24.747ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 26 reason = Timeout
4712024-12-14T01:15:24.747ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
4722024-12-14T01:15:24.747ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
4732024-12-14T01:15:24.747ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 25 reason = Timeout
4742024-12-14T01:15:24.747ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 25 reason = Timeout
4752024-12-14T01:15:24.747ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 14 reason = Timeout
4762024-12-14T01:15:24.747ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
4772024-12-14T01:15:24.747ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
4782024-12-14T01:15:24.747ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 14 reason = Timeout
4792024-12-14T01:15:24.747ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 13 reason = Timeout
4802024-12-14T01:15:24.747ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2024-12-14 01:05:24.747638515 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
4812024-12-14T01:15:24.747ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
4822024-12-14T01:15:24.747ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 29 reason = Timeout
4832024-12-14T01:15:24.747ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
4842024-12-14T01:15:24.747ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 13 reason = Timeout
4852024-12-14T01:15:24.747ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
4862024-12-14T01:15:24.747ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 14 reason = Timeout
4872024-12-14T01:15:24.747ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
4882024-12-14T01:15:24.747ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 14 reason = Timeout
4892024-12-14T01:15:24.747ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
4902024-12-14T01:15:24.747ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 25 reason = Timeout
4912024-12-14T01:15:24.747ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 73 reason = Timeout
4922024-12-14T01:15:24.747ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
4932024-12-14T01:15:24.747ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
4942024-12-14T01:15:24.747ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
4952024-12-14T01:15:24.747ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 25 reason = Timeout
4962024-12-14T01:15:24.747ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
4972024-12-14T01:15:24.747ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
4982024-12-14T01:15:24.747ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
4992024-12-14T01:15:24.747ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
5002024-12-14T01:15:24.747ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
5012024-12-14T01:15:24.747ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
5022024-12-14T01:15:24.748ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
5032024-12-14T01:15:24.748ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
5042024-12-14T01:15:24.748ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
5052024-12-14T01:15:24.748ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
5062024-12-14T01:15:24.748ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
5072024-12-14T01:15:24.748ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
5082024-12-14T01:15:24.748ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
5092024-12-14T01:15:24.748ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
5102024-12-14T01:15:24.748ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5112024-12-14T01:15:24.748ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
5122024-12-14T01:15:24.748ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
5132024-12-14T01:15:24.748ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
5142024-12-14T01:15:24.748ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
5152024-12-14T01:15:24.748ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
5162024-12-14T01:15:24.748ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
5172024-12-14T01:15:24.748ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
5182024-12-14T01:15:24.748ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
5192024-12-14T01:15:24.748ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("1b585a7c-7eb0-4894-b8b3-3ef1983891e5.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("36100ee5-fa9a-4857-a844-70bec0e8601f.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("830b88b6-26da-4988-83b1-cf4193531cd5.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("830b88b6-26da-4988-83b1-cf4193531cd5.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:2::1))) }], valid_until: Instant { tv_sec: 1487, tv_nsec: 577222026 } })
5202024-12-14T01:15:24.748ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
5212024-12-14T01:15:24.748ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
5222024-12-14T01:15:24.748ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
5232024-12-14T01:15:24.748ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("381b8112-f609-4c4d-ac80-0170ad8fc4bd.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("795aee12-3394-49fa-9cb5-487c97a85b2d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("795aee12-3394-49fa-9cb5-487c97a85b2d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::9))) }], valid_until: Instant { tv_sec: 1487, tv_nsec: 577284908 } })
5242024-12-14T01:15:24.748ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
5252024-12-14T01:15:24.748ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
5262024-12-14T01:15:24.748ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
5272024-12-14T01:15:24.748ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
5282024-12-14T01:15:24.748ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } dns_group = external
5292024-12-14T01:15:24.748ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
5302024-12-14T01:15:24.748ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
5312024-12-14T01:15:24.748ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.266157ms iteration = 14 status = {"addresses":["[fd00:1122:3344:101::8]:5353","[fd00:1122:3344:101::9]:5353"]}
5322024-12-14T01:15:24.748ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } dns_group = internal
5332024-12-14T01:15:24.748ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.204125ms iteration = 14 status = {"addresses":["[fd00:1122:3344:1::1]:5353","[fd00:1122:3344:3::1]:5353","[fd00:1122:3344:2::1]:5353"]}
5342024-12-14T01:15:24.749ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
5352024-12-14T01:15:24.750ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5362024-12-14T01:15:24.750ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.362258ms iteration = 25 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
5372024-12-14T01:15:24.751ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5382024-12-14T01:15:24.752ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
5392024-12-14T01:15:24.752ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
5402024-12-14T01:15:24.752ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
5412024-12-14T01:15:24.753ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
5422024-12-14T01:15:24.753ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
5432024-12-14T01:15:24.753ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
5442024-12-14T01:15:24.754ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5452024-12-14T01:15:24.755ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-7e423e95-464a-471f-a0d9-4799977f0b0f.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-7e423e95-464a-471f-a0d9-4799977f0b0f.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1487, tv_nsec: 584574629 } })
5462024-12-14T01:15:24.755ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1103
5472024-12-14T01:15:24.756ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
5482024-12-14T01:15:24.756ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
5492024-12-14T01:15:24.756ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 8.691501ms iteration = 25 status = {"errors":[],"records_set_to_done":[]}
5502024-12-14T01:15:24.756ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
5512024-12-14T01:15:24.756ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
5522024-12-14T01:15:24.756ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
5532024-12-14T01:15:24.758ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-7e423e95-464a-471f-a0d9-4799977f0b0f.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-7e423e95-464a-471f-a0d9-4799977f0b0f.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1487, tv_nsec: 587344109 } })
5542024-12-14T01:15:24.758ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2024-12-14 01:05:24.747638515 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
5552024-12-14T01:15:24.758ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 10.901265ms iteration = 13 status = {"expiration":"2024-12-14T01:05:24.747638515Z","pruned":[]}
5562024-12-14T01:15:24.758ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1103
5572024-12-14T01:15:24.759ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5582024-12-14T01:15:24.761ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5592024-12-14T01:15:24.763ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
5602024-12-14T01:15:24.763ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5612024-12-14T01:15:24.763ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5622024-12-14T01:15:24.763ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
5632024-12-14T01:15:24.763ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
5642024-12-14T01:15:24.763ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
5652024-12-14T01:15:24.765ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5662024-12-14T01:15:24.765ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
5672024-12-14T01:15:24.765ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 18.074573ms iteration = 13 status = {"errors":[],"found_port_ok":[]}
5682024-12-14T01:15:24.765ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
5692024-12-14T01:15:24.770ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
5702024-12-14T01:15:24.770ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
5712024-12-14T01:15:24.770ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
5722024-12-14T01:15:24.772ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5732024-12-14T01:15:24.775ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
5742024-12-14T01:15:24.775ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
5752024-12-14T01:15:24.775ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
5762024-12-14T01:15:24.776ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5772024-12-14T01:15:24.778ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5782024-12-14T01:15:24.781ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
5792024-12-14T01:15:24.781ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
5802024-12-14T01:15:24.781ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
5812024-12-14T01:15:24.781ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
5822024-12-14T01:15:24.781ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
5832024-12-14T01:15:24.781ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
5842024-12-14T01:15:24.782ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5852024-12-14T01:15:24.783ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5862024-12-14T01:15:24.783ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5872024-12-14T01:15:24.784ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
5882024-12-14T01:15:24.784ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 37.249197ms iteration = 25 status = {"errors":[],"garbage_collect_requested":[]}
5892024-12-14T01:15:24.784ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
5902024-12-14T01:15:24.787ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
5912024-12-14T01:15:24.788ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
5922024-12-14T01:15:24.788ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
5932024-12-14T01:15:24.788ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
5942024-12-14T01:15:24.788ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
5952024-12-14T01:15:24.789ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5962024-12-14T01:15:24.789ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
5972024-12-14T01:15:24.789ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
5982024-12-14T01:15:24.790ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
5992024-12-14T01:15:24.790ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
6002024-12-14T01:15:24.791ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 8770bfcc-3a71-4738-a6b0-91008cadcb0f, lookup_type: ById(8770bfcc-3a71-4738-a6b0-91008cadcb0f) } result = Ok(())
6012024-12-14T01:15:24.791ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6022024-12-14T01:15:24.791ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6032024-12-14T01:15:24.792ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6042024-12-14T01:15:24.793ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
6052024-12-14T01:15:24.793ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
6062024-12-14T01:15:24.794ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
6072024-12-14T01:15:24.794ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
6082024-12-14T01:15:24.794ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
6092024-12-14T01:15:24.799ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6102024-12-14T01:15:24.801ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
6112024-12-14T01:15:24.801ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:171 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
6122024-12-14T01:15:24.801ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 53.518859ms iteration = 13 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
6132024-12-14T01:15:24.801ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6142024-12-14T01:15:24.802ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
6152024-12-14T01:15:24.802ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
6162024-12-14T01:15:24.802ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
6172024-12-14T01:15:24.804ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
6182024-12-14T01:15:24.804ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
6192024-12-14T01:15:24.804ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
6202024-12-14T01:15:24.804ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6212024-12-14T01:15:24.804ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
6222024-12-14T01:15:24.804ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
6232024-12-14T01:15:24.805ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6242024-12-14T01:15:24.807ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
6252024-12-14T01:15:24.807ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
6262024-12-14T01:15:24.807ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
6272024-12-14T01:15:24.807ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6282024-12-14T01:15:24.808ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-7e423e95-464a-471f-a0d9-4799977f0b0f.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-7e423e95-464a-471f-a0d9-4799977f0b0f.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1487, tv_nsec: 637322595 } })
6292024-12-14T01:15:24.808ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1103 rack_id = f30fd360-30ae-4088-9821-aa98c2bc8c09
6302024-12-14T01:15:24.809ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
6312024-12-14T01:15:24.809ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
6322024-12-14T01:15:24.809ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
6332024-12-14T01:15:24.810ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
6342024-12-14T01:15:24.810ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
6352024-12-14T01:15:24.811ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
6362024-12-14T01:15:24.811ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
6372024-12-14T01:15:24.811ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
6382024-12-14T01:15:24.811ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = dc14d234-dd1b-4489-95ad-e2a18f206bb7 roles = RoleSet { roles: {} } uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
6392024-12-14T01:15:24.812ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = dc14d234-dd1b-4489-95ad-e2a18f206bb7 resource = Database result = Ok(()) uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
6402024-12-14T01:15:24.812ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
6412024-12-14T01:15:24.813ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
6422024-12-14T01:15:24.814ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
6432024-12-14T01:15:24.814ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
6442024-12-14T01:15:24.817ZDEBGnexus (BackgroundTasks): activation complete background_task = crdb_node_id_collector elapsed = 69.666126ms iteration = 6 status = {"nsuccess":5}
6452024-12-14T01:15:24.817ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
6462024-12-14T01:15:24.817ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:140 found = 0 not_ready_to_be_deleted = 0
6472024-12-14T01:15:24.817ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 70.472059ms iteration = 13 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
6482024-12-14T01:15:24.818ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 6355 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = dc14d234-dd1b-4489-95ad-e2a18f206bb7 response_code = 200 uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
6492024-12-14T01:15:24.822ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 8770bfcc-3a71-4738-a6b0-91008cadcb0f original_time_created = 2024-12-14 01:03:15.378067 UTC target_id = 8770bfcc-3a71-4738-a6b0-91008cadcb0f time_created = 2024-12-14 01:03:15.378067 UTC
6502024-12-14T01:15:24.823ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 75.549876ms iteration = 73 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"8770bfcc-3a71-4738-a6b0-91008cadcb0f","time_created":"2024-12-14T01:03:15.378067Z"}
6512024-12-14T01:15:24.829ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
6522024-12-14T01:15:24.829ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
6532024-12-14T01:15:24.830ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
6542024-12-14T01:15:24.830ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 83.335332ms iteration = 25 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
6552024-12-14T01:15:24.831ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
6562024-12-14T01:15:24.831ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
6572024-12-14T01:15:24.841ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6582024-12-14T01:15:24.842ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
6592024-12-14T01:15:24.842ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
6602024-12-14T01:15:24.842ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
6612024-12-14T01:15:24.842ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
6622024-12-14T01:15:24.842ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
6632024-12-14T01:15:24.845ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
6642024-12-14T01:15:24.847ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
6652024-12-14T01:15:24.848ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
6662024-12-14T01:15:24.848ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
6672024-12-14T01:15:24.848ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
6682024-12-14T01:15:24.849ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
6692024-12-14T01:15:24.849ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
6702024-12-14T01:15:24.849ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
6712024-12-14T01:15:24.849ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
6722024-12-14T01:15:24.850ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 103.191937ms iteration = 25 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[]}
6732024-12-14T01:15:24.851ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6742024-12-14T01:15:24.852ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
6752024-12-14T01:15:24.852ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
6762024-12-14T01:15:24.854ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
6772024-12-14T01:15:24.854ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
6782024-12-14T01:15:24.854ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
6792024-12-14T01:15:24.855ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
6802024-12-14T01:15:24.855ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
6812024-12-14T01:15:24.856ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 108.950033ms iteration = 73 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
6822024-12-14T01:15:24.857ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6832024-12-14T01:15:24.859ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 7e423e95-464a-471f-a0d9-4799977f0b0f, lookup_type: ById(7e423e95-464a-471f-a0d9-4799977f0b0f) } result = Ok(())
6842024-12-14T01:15:24.859ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
6852024-12-14T01:15:24.859ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
6862024-12-14T01:15:24.861ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 113.585507ms iteration = 25 status = 0
6872024-12-14T01:15:24.867ZINFOnexus (BackgroundTasks): found Silo/DNS/TLS config (changed) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2024-12-14T01:03:19.260831Z, time_modified: 2024-12-14T01:03:19.260831Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 2c668072-a9a1-4a69-928c-e118c99521be, db_silo: Silo { identity: SiloIdentity { id: 2c668072-a9a1-4a69-928c-e118c99521be, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-12-14T01:03:52.950638Z, time_modified: 2024-12-14T01:03:52.950638Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "ad8fd43e0fc4f6284dc5e0b0d0f2e6a6e8d979d13f86a71c9b81d6817e439efd" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 2c668072-a9a1-4a69-928c-e118c99521be, db_silo: Silo { identity: SiloIdentity { id: 2c668072-a9a1-4a69-928c-e118c99521be, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-12-14T01:03:52.950638Z, time_modified: 2024-12-14T01:03:52.950638Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "ad8fd43e0fc4f6284dc5e0b0d0f2e6a6e8d979d13f86a71c9b81d6817e439efd" }] }) } file = nexus/src/app/background/tasks/external_endpoints.rs:94
6882024-12-14T01:15:24.867ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 119.967492ms iteration = 14 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"2c668072-a9a1-4a69-928c-e118c99521be","tls_certs":["ad8fd43e0fc4f6284dc5e0b0d0f2e6a6e8d979d13f86a71c9b81d6817e439efd"]}},"default_endpoint":{"silo_id":"2c668072-a9a1-4a69-928c-e118c99521be","tls_certs":["ad8fd43e0fc4f6284dc5e0b0d0f2e6a6e8d979d13f86a71c9b81d6817e439efd"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
6892024-12-14T01:15:24.867ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 120.512338ms iteration = 25 status = {"errors":[],"requests_created_ok":[],"start_invoked_ok":[]}
6902024-12-14T01:15:24.869ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 121.483065ms iteration = 25 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
6912024-12-14T01:15:24.870ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6922024-12-14T01:15:24.873ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
6932024-12-14T01:15:24.873ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
6942024-12-14T01:15:24.873ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
6952024-12-14T01:15:24.873ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6962024-12-14T01:15:24.876ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
6972024-12-14T01:15:24.876ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
6982024-12-14T01:15:24.876ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
6992024-12-14T01:15:24.876ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
7002024-12-14T01:15:24.876ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
7012024-12-14T01:15:24.877ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
7022024-12-14T01:15:24.877ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
7032024-12-14T01:15:24.877ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
7042024-12-14T01:15:24.877ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
7052024-12-14T01:15:24.877ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
7062024-12-14T01:15:24.878ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "d105a55c-5e14-4ee2-aefb-3e9be762ddf4", "content-length": "26", "date": "Sat, 14 Dec 2024 01:15:24 GMT"} })
7072024-12-14T01:15:24.878ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
7082024-12-14T01:15:24.878ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
7092024-12-14T01:15:24.878ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7102024-12-14T01:15:24.878ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
7112024-12-14T01:15:24.878ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "5a1f1a23-108b-4a4a-b8b7-d389bb1a6c39", "content-length": "26", "date": "Sat, 14 Dec 2024 01:15:24 GMT"} })
7122024-12-14T01:15:24.878ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
7132024-12-14T01:15:24.879ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
7142024-12-14T01:15:24.879ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = d9c36d0c-d7d2-439f-a748-798b6d8dea0c dns_zone_name = oxide.test found_so_far = 0 version = 2
7152024-12-14T01:15:24.879ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
7162024-12-14T01:15:24.879ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
7172024-12-14T01:15:24.880ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
7182024-12-14T01:15:24.880ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
7192024-12-14T01:15:24.880ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 133.294417ms iteration = 25 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
7202024-12-14T01:15:24.881ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
7212024-12-14T01:15:24.881ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7222024-12-14T01:15:24.881ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
7232024-12-14T01:15:24.881ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
7242024-12-14T01:15:24.881ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
7252024-12-14T01:15:24.882ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
7262024-12-14T01:15:24.886ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
7272024-12-14T01:15:24.886ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
7282024-12-14T01:15:24.886ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
7292024-12-14T01:15:24.910ZWARNnexus (BackgroundTasks): instance reincarnation completed with errors background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:103 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0 query_errors = 0 restart_errors = 0
7302024-12-14T01:15:24.910ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 163.042338ms iteration = 13 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
7312024-12-14T01:15:24.926ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
7322024-12-14T01:15:24.926ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 866c9448-c447-4b92-96b6-611d51967e09 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
7332024-12-14T01:15:24.926ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
7342024-12-14T01:15:24.926ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
7352024-12-14T01:15:24.935ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
7362024-12-14T01:15:24.935ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
7372024-12-14T01:15:24.970ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1111 rack_id = f30fd360-30ae-4088-9821-aa98c2bc8c09 zone_address = fd00:1122:3344:101::2
7382024-12-14T01:15:24.970ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = f30fd360-30ae-4088-9821-aa98c2bc8c09 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
7392024-12-14T01:15:24.970ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = f30fd360-30ae-4088-9821-aa98c2bc8c09 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "eaf4d1e9-7d0d-4c4a-ac24-2b84280742ea", "content-length": "26", "date": "Sat, 14 Dec 2024 01:15:24 GMT"} })
7402024-12-14T01:15:24.970ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1116 rack_id = f30fd360-30ae-4088-9821-aa98c2bc8c09 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
7412024-12-14T01:15:24.972ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = f30fd360-30ae-4088-9821-aa98c2bc8c09
7422024-12-14T01:15:24.982ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 234.674351ms iteration = 29 status = {"physical_disks_added":0}
7432024-12-14T01:15:25.016ZINFOnexus (BackgroundTasks): VPC route manager sled 7e423e95-464a-471f-a0d9-4799977f0b0f background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
7442024-12-14T01:15:25.016ZDEBGnexus (BackgroundTasks): client request SledAgent = 7e423e95-464a-471f-a0d9-4799977f0b0f background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
7452024-12-14T01:15:25.017ZDEBGnexus (BackgroundTasks): client response SledAgent = 7e423e95-464a-471f-a0d9-4799977f0b0f background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "80c0f68d-abdf-430d-a56b-288dd20fae18", "content-length": "372", "date": "Sat, 14 Dec 2024 01:15:25 GMT"} })
7462024-12-14T01:15:25.017ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
7472024-12-14T01:15:25.017ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
7482024-12-14T01:15:25.018ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = d9c36d0c-d7d2-439f-a748-798b6d8dea0c dns_zone_name = oxide.test found_so_far = 1 version = 2
7492024-12-14T01:15:25.018ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
7502024-12-14T01:15:25.018ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2024-12-14 01:03:53.017536 UTC dns_group = external generation = 2
7512024-12-14T01:15:25.018ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 271.563609ms iteration = 14 status = {"generation":2}
7522024-12-14T01:15:25.019ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7532024-12-14T01:15:25.019ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7542024-12-14T01:15:25.022ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
7552024-12-14T01:15:25.022ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
7562024-12-14T01:15:25.022ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
7572024-12-14T01:15:25.023ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
7582024-12-14T01:15:25.023ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
7592024-12-14T01:15:25.023ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
7602024-12-14T01:15:25.025ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 866c9448-c447-4b92-96b6-611d51967e09 dns_zone_name = control-plane.oxide.internal found_so_far = 51 version = 1
7612024-12-14T01:15:25.025ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7622024-12-14T01:15:25.025ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
7632024-12-14T01:15:25.025ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2024-12-14 01:03:51.993091 UTC dns_group = internal generation = 1
7642024-12-14T01:15:25.025ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 277.638025ms iteration = 14 status = {"generation":1}
7652024-12-14T01:15:25.027ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
7662024-12-14T01:15:25.027ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
7672024-12-14T01:15:25.027ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
7682024-12-14T01:15:25.027ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
7692024-12-14T01:15:25.032ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
7702024-12-14T01:15:25.032ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "4c3044a8-56be-4b2a-9fef-9655b46248b9", "date": "Sat, 14 Dec 2024 01:15:25 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
7712024-12-14T01:15:25.032ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
7722024-12-14T01:15:25.033ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = buskin
7732024-12-14T01:15:25.033ZDEBGnexus (BackgroundTasks): client request SledAgent = 7e423e95-464a-471f-a0d9-4799977f0b0f background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
7742024-12-14T01:15:25.033ZDEBGnexus (BackgroundTasks): client response SledAgent = 7e423e95-464a-471f-a0d9-4799977f0b0f background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "b5e86f40-ac96-4bbc-9a0c-e0e0d579dbbb", "date": "Sat, 14 Dec 2024 01:15:25 GMT"} })
7752024-12-14T01:15:25.033ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
7762024-12-14T01:15:25.033ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
7772024-12-14T01:15:25.033ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "624f86a4-a62e-4bef-9f50-efbcd0b0e2e4", "date": "Sat, 14 Dec 2024 01:15:25 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
7782024-12-14T01:15:25.033ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
7792024-12-14T01:15:25.035ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
7802024-12-14T01:15:25.035ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
7812024-12-14T01:15:25.036ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
7822024-12-14T01:15:25.036ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 288.737636ms iteration = 15 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
7832024-12-14T01:15:25.038ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
7842024-12-14T01:15:25.039ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
7852024-12-14T01:15:25.041ZINFOnexus (BackgroundTasks): VPC route manager sled 7e423e95-464a-471f-a0d9-4799977f0b0f push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
7862024-12-14T01:15:25.041ZDEBGnexus (BackgroundTasks): client request SledAgent = 7e423e95-464a-471f-a0d9-4799977f0b0f background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
7872024-12-14T01:15:25.041ZDEBGnexus (BackgroundTasks): client response SledAgent = 7e423e95-464a-471f-a0d9-4799977f0b0f background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "99bc5905-3e86-444a-b6c9-caea90bb8e1e", "date": "Sat, 14 Dec 2024 01:15:25 GMT"} })
7882024-12-14T01:15:25.043ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 296.046867ms iteration = 27 status = {}
7892024-12-14T01:15:25.058ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 311.101863ms iteration = 25 status = {}
7902024-12-14T01:15:25.109ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
7912024-12-14T01:15:25.109ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "9abea522-22c8-4de2-9f33-e98e6162bc4f", "content-length": "2", "date": "Sat, 14 Dec 2024 01:15:24 GMT"} })
7922024-12-14T01:15:25.110ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
7932024-12-14T01:15:25.111ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
7942024-12-14T01:15:25.113ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 365.746744ms iteration = 25 status = null
7952024-12-14T01:15:25.141ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
7962024-12-14T01:15:25.142ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "bea1f2be-9a42-44e9-8a37-6f31c2f4c3a5", "date": "Sat, 14 Dec 2024 01:15:25 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
7972024-12-14T01:15:25.142ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
7982024-12-14T01:15:25.144ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "a36b5a83-ebea-439a-a988-9f2b81c76107", "date": "Sat, 14 Dec 2024 01:15:25 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
7992024-12-14T01:15:25.144ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
8002024-12-14T01:15:25.145ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "4e124b48-cbd4-4911-bb53-ac10b41ac80a", "date": "Sat, 14 Dec 2024 01:15:25 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
8012024-12-14T01:15:25.145ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
8022024-12-14T01:15:25.147ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
8032024-12-14T01:15:25.147ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 399.772079ms iteration = 15 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
8042024-12-14T01:15:25.221ZDEBGnexus (BackgroundTasks): client request SledAgent = 7e423e95-464a-471f-a0d9-4799977f0b0f background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
8052024-12-14T01:15:25.222ZDEBGnexus (BackgroundTasks): client response SledAgent = 7e423e95-464a-471f-a0d9-4799977f0b0f background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "4f53a73a-ac83-42d8-aa54-ea55e3e92501", "content-length": "691", "date": "Sat, 14 Dec 2024 01:15:25 GMT"} })
8062024-12-14T01:15:25.222ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = buskin
8072024-12-14T01:15:25.222ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = buskin
8082024-12-14T01:15:25.222ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = buskin
8092024-12-14T01:15:25.224ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 476.914832ms iteration = 26 status = {}
8102024-12-14T01:15:25.281ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
8112024-12-14T01:15:25.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
8122024-12-14T01:15:25.286ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: bb1a4b50-730e-48ee-a311-8a9b9d7bcb73, rack_id: f30fd360-30ae-4088-9821-aa98c2bc8c09, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(bb9f5a1a-5708-4d58-ba4f-c62102236426) } rack_id = f30fd360-30ae-4088-9821-aa98c2bc8c09 switch_location = Switch0
8132024-12-14T01:15:25.286ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
8142024-12-14T01:15:25.287ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
8152024-12-14T01:15:25.318ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = f30fd360-30ae-4088-9821-aa98c2bc8c09 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
8162024-12-14T01:15:25.319ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = f30fd360-30ae-4088-9821-aa98c2bc8c09 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "13b09724-47f7-42f2-83bc-f61ebcac1ee4", "content-length": "134", "date": "Sat, 14 Dec 2024 01:15:24 GMT"} })
8172024-12-14T01:15:25.319ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1746 rack_id = f30fd360-30ae-4088-9821-aa98c2bc8c09 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
8182024-12-14T01:15:25.319ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1773 rack_id = f30fd360-30ae-4088-9821-aa98c2bc8c09 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = bb9f5a1a-5708-4d58-ba4f-c62102236426
8192024-12-14T01:15:25.319ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = f30fd360-30ae-4088-9821-aa98c2bc8c09 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
8202024-12-14T01:15:25.319ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = f30fd360-30ae-4088-9821-aa98c2bc8c09 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
8212024-12-14T01:15:25.319ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = f30fd360-30ae-4088-9821-aa98c2bc8c09 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
8222024-12-14T01:15:25.319ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = f30fd360-30ae-4088-9821-aa98c2bc8c09 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
8232024-12-14T01:15:25.319ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = f30fd360-30ae-4088-9821-aa98c2bc8c09 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
8242024-12-14T01:15:25.319ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1920 rack_id = f30fd360-30ae-4088-9821-aa98c2bc8c09 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
8252024-12-14T01:15:25.319ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = f30fd360-30ae-4088-9821-aa98c2bc8c09 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
8262024-12-14T01:15:25.319ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = f30fd360-30ae-4088-9821-aa98c2bc8c09 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
8272024-12-14T01:15:25.320ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = f30fd360-30ae-4088-9821-aa98c2bc8c09
8282024-12-14T01:15:25.320ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
8292024-12-14T01:15:25.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
8302024-12-14T01:15:25.323ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = f30fd360-30ae-4088-9821-aa98c2bc8c09 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
8312024-12-14T01:15:25.323ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = f30fd360-30ae-4088-9821-aa98c2bc8c09 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "d10f616a-ab02-4bd1-9a3e-f057b98ed225", "content-length": "2", "date": "Sat, 14 Dec 2024 01:15:24 GMT"} })
8322024-12-14T01:15:25.323ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = f30fd360-30ae-4088-9821-aa98c2bc8c09 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
8332024-12-14T01:15:25.323ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = f30fd360-30ae-4088-9821-aa98c2bc8c09 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "20ddeeb5-8505-4d38-93a8-dd37e5c52e1d", "content-length": "52", "date": "Sat, 14 Dec 2024 01:15:24 GMT"} })
8342024-12-14T01:15:25.323ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.1.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = f30fd360-30ae-4088-9821-aa98c2bc8c09 switch_location = Switch0
8352024-12-14T01:15:25.323ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = f30fd360-30ae-4088-9821-aa98c2bc8c09 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
8362024-12-14T01:15:25.434ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = f30fd360-30ae-4088-9821-aa98c2bc8c09 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "2f3b932e-f572-4118-b99b-4d02611ef2ec", "date": "Sat, 14 Dec 2024 01:15:25 GMT"} })
8372024-12-14T01:15:25.434ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = f30fd360-30ae-4088-9821-aa98c2bc8c09 switch = Switch0
8382024-12-14T01:15:25.434ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = f30fd360-30ae-4088-9821-aa98c2bc8c09 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
8392024-12-14T01:15:25.435ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = f30fd360-30ae-4088-9821-aa98c2bc8c09 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "ee623aca-6e1f-4b2d-9805-66b390f3e295", "content-length": "682", "date": "Sat, 14 Dec 2024 01:15:25 GMT"} })
8402024-12-14T01:15:25.435ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
8412024-12-14T01:15:25.435ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
8422024-12-14T01:15:25.440ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
8432024-12-14T01:15:25.440ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
8442024-12-14T01:15:25.443ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
8452024-12-14T01:15:25.444ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
8462024-12-14T01:15:25.446ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
8472024-12-14T01:15:25.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
8482024-12-14T01:15:25.454ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 706.594479ms iteration = 25 status = {}
8492024-12-14T01:15:25.846ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 0ec52038-a7c0-4067-9529-b06de9f94d30 roles = RoleSet { roles: {} } uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
8502024-12-14T01:15:25.846ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 0ec52038-a7c0-4067-9529-b06de9f94d30 resource = Database result = Ok(()) uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
8512024-12-14T01:15:25.848ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2465 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 0ec52038-a7c0-4067-9529-b06de9f94d30 response_code = 200 uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
8522024-12-14T01:15:26.868ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = b38d0ec5-0aa9-40de-bea9-a5dc28cfd337 roles = RoleSet { roles: {} } uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
8532024-12-14T01:15:26.869ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = b38d0ec5-0aa9-40de-bea9-a5dc28cfd337 resource = Database result = Ok(()) uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
8542024-12-14T01:15:26.871ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2331 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = b38d0ec5-0aa9-40de-bea9-a5dc28cfd337 response_code = 200 uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
8552024-12-14T01:15:27.889ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 1daca1a2-bd6c-4d2c-8d10-c4b1831d350b roles = RoleSet { roles: {} } uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
8562024-12-14T01:15:27.890ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 1daca1a2-bd6c-4d2c-8d10-c4b1831d350b resource = Database result = Ok(()) uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
8572024-12-14T01:15:27.891ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2054 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 1daca1a2-bd6c-4d2c-8d10-c4b1831d350b response_code = 200 uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
8582024-12-14T01:15:28.911ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 67daa819-e843-494c-88a4-8f8ff2692ab0 roles = RoleSet { roles: {} } uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
8592024-12-14T01:15:28.911ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 67daa819-e843-494c-88a4-8f8ff2692ab0 resource = Database result = Ok(()) uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
8602024-12-14T01:15:28.913ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2510 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 67daa819-e843-494c-88a4-8f8ff2692ab0 response_code = 200 uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
8612024-12-14T01:15:29.932ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 410da6be-50a6-4353-9f66-87441f8e292f roles = RoleSet { roles: {} } uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
8622024-12-14T01:15:29.932ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 410da6be-50a6-4353-9f66-87441f8e292f resource = Database result = Ok(()) uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
8632024-12-14T01:15:29.934ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2330 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 410da6be-50a6-4353-9f66-87441f8e292f response_code = 200 uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
8642024-12-14T01:15:30.955ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = be32d97f-bda6-4450-bb7a-2733ccd2f8a1 roles = RoleSet { roles: {} } uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
8652024-12-14T01:15:30.955ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = be32d97f-bda6-4450-bb7a-2733ccd2f8a1 resource = Database result = Ok(()) uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
8662024-12-14T01:15:30.957ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2458 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = be32d97f-bda6-4450-bb7a-2733ccd2f8a1 response_code = 200 uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
8672024-12-14T01:15:31.978ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 22948b49-125e-4ee9-bfaa-035a216ddce3 roles = RoleSet { roles: {} } uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
8682024-12-14T01:15:31.979ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 22948b49-125e-4ee9-bfaa-035a216ddce3 resource = Database result = Ok(()) uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
8692024-12-14T01:15:31.980ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2072 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 22948b49-125e-4ee9-bfaa-035a216ddce3 response_code = 200 uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
8702024-12-14T01:15:32.999ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 8c3ee11d-1081-4a2f-a286-d3259601e0b1 roles = RoleSet { roles: {} } uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
8712024-12-14T01:15:33.000ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 8c3ee11d-1081-4a2f-a286-d3259601e0b1 resource = Database result = Ok(()) uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
8722024-12-14T01:15:33.001ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2073 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 8c3ee11d-1081-4a2f-a286-d3259601e0b1 response_code = 200 uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
8732024-12-14T01:15:33.462ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1471 local_addr = [fd00:1122:3344:101::c]:54941 method = GET remote_addr = [fd00:1122:3344:101::d]:43526 req_id = 8a10c04c-bcb4-4090-a5d2-6f2180b219f5 response_code = 200 uri = /499bc009-7d23-4916-9be8-0a6a8f4e2f93
8742024-12-14T01:15:34.021ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 67481b41-358f-41eb-8562-176c8e17f4cc roles = RoleSet { roles: {} } uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
8752024-12-14T01:15:34.022ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 67481b41-358f-41eb-8562-176c8e17f4cc resource = Database result = Ok(()) uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
8762024-12-14T01:15:34.024ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2310 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 67481b41-358f-41eb-8562-176c8e17f4cc response_code = 200 uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
8772024-12-14T01:15:34.746ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 74 reason = Timeout
8782024-12-14T01:15:34.746ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
8792024-12-14T01:15:34.746ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 74 reason = Timeout
8802024-12-14T01:15:34.746ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
8812024-12-14T01:15:34.746ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
8822024-12-14T01:15:34.746ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
8832024-12-14T01:15:34.748ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8842024-12-14T01:15:34.749ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
8852024-12-14T01:15:34.749ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
8862024-12-14T01:15:34.751ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
8872024-12-14T01:15:34.751ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
8882024-12-14T01:15:34.751ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
8892024-12-14T01:15:34.753ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
8902024-12-14T01:15:34.753ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
8912024-12-14T01:15:34.755ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8922024-12-14T01:15:34.757ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 8770bfcc-3a71-4738-a6b0-91008cadcb0f, lookup_type: ById(8770bfcc-3a71-4738-a6b0-91008cadcb0f) } result = Ok(())
8932024-12-14T01:15:34.757ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
8942024-12-14T01:15:34.757ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
8952024-12-14T01:15:34.770ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 8770bfcc-3a71-4738-a6b0-91008cadcb0f original_time_created = 2024-12-14 01:03:15.378067 UTC target_id = 8770bfcc-3a71-4738-a6b0-91008cadcb0f time_created = 2024-12-14 01:03:15.378067 UTC
8962024-12-14T01:15:34.770ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 23.788618ms iteration = 74 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"8770bfcc-3a71-4738-a6b0-91008cadcb0f","time_created":"2024-12-14T01:03:15.378067Z"}
8972024-12-14T01:15:34.778ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 31.910043ms iteration = 74 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
8982024-12-14T01:15:35.045ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 72d9dd91-ef24-4fcc-8033-496e187962bf roles = RoleSet { roles: {} } uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
8992024-12-14T01:15:35.045ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 72d9dd91-ef24-4fcc-8033-496e187962bf resource = Database result = Ok(()) uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
9002024-12-14T01:15:35.047ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2233 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 72d9dd91-ef24-4fcc-8033-496e187962bf response_code = 200 uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
9012024-12-14T01:15:36.067ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = af092b4f-3821-4419-b2c8-869aff5ad21f roles = RoleSet { roles: {} } uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
9022024-12-14T01:15:36.067ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = af092b4f-3821-4419-b2c8-869aff5ad21f resource = Database result = Ok(()) uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
9032024-12-14T01:15:36.069ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2370 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = af092b4f-3821-4419-b2c8-869aff5ad21f response_code = 200 uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
9042024-12-14T01:15:37.089ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 718dba3b-5711-47ec-be18-e44feb85dab7 roles = RoleSet { roles: {} } uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
9052024-12-14T01:15:37.090ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 718dba3b-5711-47ec-be18-e44feb85dab7 resource = Database result = Ok(()) uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
9062024-12-14T01:15:37.091ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2389 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 718dba3b-5711-47ec-be18-e44feb85dab7 response_code = 200 uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
9072024-12-14T01:15:38.112ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = bf1490fb-9f0a-465c-b281-8e0bdbd51a49 roles = RoleSet { roles: {} } uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
9082024-12-14T01:15:38.112ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = bf1490fb-9f0a-465c-b281-8e0bdbd51a49 resource = Database result = Ok(()) uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
9092024-12-14T01:15:38.114ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2450 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = bf1490fb-9f0a-465c-b281-8e0bdbd51a49 response_code = 200 uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
9102024-12-14T01:15:39.133ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = d96e4f5c-2d8d-46fa-961a-b5a10539857e roles = RoleSet { roles: {} } uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
9112024-12-14T01:15:39.134ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = d96e4f5c-2d8d-46fa-961a-b5a10539857e resource = Database result = Ok(()) uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
9122024-12-14T01:15:39.136ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2231 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = d96e4f5c-2d8d-46fa-961a-b5a10539857e response_code = 200 uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
9132024-12-14T01:15:40.155ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 29803f89-2a1c-431c-8d8a-225a28e9f1b2 roles = RoleSet { roles: {} } uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
9142024-12-14T01:15:40.155ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 29803f89-2a1c-431c-8d8a-225a28e9f1b2 resource = Database result = Ok(()) uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
9152024-12-14T01:15:40.157ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2113 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 29803f89-2a1c-431c-8d8a-225a28e9f1b2 response_code = 200 uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
9162024-12-14T01:15:41.176ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = d0eb0486-5635-4440-b603-de60098d096d roles = RoleSet { roles: {} } uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
9172024-12-14T01:15:41.177ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = d0eb0486-5635-4440-b603-de60098d096d resource = Database result = Ok(()) uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
9182024-12-14T01:15:41.179ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2545 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = d0eb0486-5635-4440-b603-de60098d096d response_code = 200 uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
9192024-12-14T01:15:42.198ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 1d62b87b-3972-4762-ab22-fd9391e9151d roles = RoleSet { roles: {} } uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
9202024-12-14T01:15:42.198ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 1d62b87b-3972-4762-ab22-fd9391e9151d resource = Database result = Ok(()) uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
9212024-12-14T01:15:42.200ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1919 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 1d62b87b-3972-4762-ab22-fd9391e9151d response_code = 200 uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
9222024-12-14T01:15:43.220ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = ea2f614e-0ff8-478b-aca8-ebe2fc10c96a roles = RoleSet { roles: {} } uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
9232024-12-14T01:15:43.220ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = ea2f614e-0ff8-478b-aca8-ebe2fc10c96a resource = Database result = Ok(()) uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
9242024-12-14T01:15:43.222ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1957 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = ea2f614e-0ff8-478b-aca8-ebe2fc10c96a response_code = 200 uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
9252024-12-14T01:15:43.462ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1357 local_addr = [fd00:1122:3344:101::c]:54941 method = GET remote_addr = [fd00:1122:3344:101::d]:43526 req_id = 7d9a01db-0552-40f6-a769-94a8127e69a3 response_code = 200 uri = /499bc009-7d23-4916-9be8-0a6a8f4e2f93
9262024-12-14T01:15:44.240ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 46f657fc-6e82-4be2-846d-e33621463906 roles = RoleSet { roles: {} } uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
9272024-12-14T01:15:44.240ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 46f657fc-6e82-4be2-846d-e33621463906 resource = Database result = Ok(()) uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
9282024-12-14T01:15:44.242ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2376 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 46f657fc-6e82-4be2-846d-e33621463906 response_code = 200 uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
9292024-12-14T01:15:44.746ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 75 reason = Timeout
9302024-12-14T01:15:44.746ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 75 reason = Timeout
9312024-12-14T01:15:44.746ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
9322024-12-14T01:15:44.746ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9332024-12-14T01:15:44.747ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
9342024-12-14T01:15:44.747ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
9352024-12-14T01:15:44.749ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
9362024-12-14T01:15:44.749ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9372024-12-14T01:15:44.749ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
9382024-12-14T01:15:44.751ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 4.293424ms iteration = 75 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
9392024-12-14T01:15:44.751ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
9402024-12-14T01:15:44.751ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9412024-12-14T01:15:44.752ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
9422024-12-14T01:15:44.754ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9432024-12-14T01:15:44.754ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
9442024-12-14T01:15:44.756ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9452024-12-14T01:15:44.758ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 8770bfcc-3a71-4738-a6b0-91008cadcb0f, lookup_type: ById(8770bfcc-3a71-4738-a6b0-91008cadcb0f) } result = Ok(())
9462024-12-14T01:15:44.758ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9472024-12-14T01:15:44.758ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
9482024-12-14T01:15:44.767ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 8770bfcc-3a71-4738-a6b0-91008cadcb0f original_time_created = 2024-12-14 01:03:15.378067 UTC target_id = 8770bfcc-3a71-4738-a6b0-91008cadcb0f time_created = 2024-12-14 01:03:15.378067 UTC
9492024-12-14T01:15:44.767ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 21.066989ms iteration = 75 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"8770bfcc-3a71-4738-a6b0-91008cadcb0f","time_created":"2024-12-14T01:03:15.378067Z"}
9502024-12-14T01:15:45.262ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 2db325f5-45f0-4264-ac33-0862faf37a2d roles = RoleSet { roles: {} } uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
9512024-12-14T01:15:45.262ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 2db325f5-45f0-4264-ac33-0862faf37a2d resource = Database result = Ok(()) uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
9522024-12-14T01:15:45.265ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2879 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 2db325f5-45f0-4264-ac33-0862faf37a2d response_code = 200 uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
9532024-12-14T01:15:46.284ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 06a1a0a9-dc94-40ae-b76a-e3482f265aae roles = RoleSet { roles: {} } uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
9542024-12-14T01:15:46.285ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 06a1a0a9-dc94-40ae-b76a-e3482f265aae resource = Database result = Ok(()) uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
9552024-12-14T01:15:46.287ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2308 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 06a1a0a9-dc94-40ae-b76a-e3482f265aae response_code = 200 uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
9562024-12-14T01:15:47.307ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = ddf07acd-810a-4fbe-96f9-cd11aae04700 roles = RoleSet { roles: {} } uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
9572024-12-14T01:15:47.307ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = ddf07acd-810a-4fbe-96f9-cd11aae04700 resource = Database result = Ok(()) uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
9582024-12-14T01:15:47.309ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2473 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = ddf07acd-810a-4fbe-96f9-cd11aae04700 response_code = 200 uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
9592024-12-14T01:15:48.329ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = a74e76c6-1df1-40c7-ad2b-1ba4c23b7012 roles = RoleSet { roles: {} } uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
9602024-12-14T01:15:48.329ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = a74e76c6-1df1-40c7-ad2b-1ba4c23b7012 resource = Database result = Ok(()) uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
9612024-12-14T01:15:48.331ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2330 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = a74e76c6-1df1-40c7-ad2b-1ba4c23b7012 response_code = 200 uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
9622024-12-14T01:15:48.452ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:57045 req_id = dd4547ec-f9b2-4525-a1b7-cb805247ef4f roles = RoleSet { roles: {} } uri = /metrics/collectors/6e1e4a25-c6a7-49c5-bb1d-86638130bef7/producers?sort_by=id_ascending
9632024-12-14T01:15:48.452ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:57045 req_id = dd4547ec-f9b2-4525-a1b7-cb805247ef4f resource = Database result = Ok(()) uri = /metrics/collectors/6e1e4a25-c6a7-49c5-bb1d-86638130bef7/producers?sort_by=id_ascending
9642024-12-14T01:15:48.455ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 3225 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:57045 req_id = dd4547ec-f9b2-4525-a1b7-cb805247ef4f response_code = 200 uri = /metrics/collectors/6e1e4a25-c6a7-49c5-bb1d-86638130bef7/producers?sort_by=id_ascending
9652024-12-14T01:15:48.455ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:57045 req_id = 6edcd001-2663-4a3e-a8fb-1998e30a7027 roles = RoleSet { roles: {} } uri = /metrics/collectors/6e1e4a25-c6a7-49c5-bb1d-86638130bef7/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmNGM2ZGNlZi0yNTc3LTQxMmUtODNkMS0zNzkyZGNjMDQ2MWQifX0%3D
9662024-12-14T01:15:48.456ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:57045 req_id = 6edcd001-2663-4a3e-a8fb-1998e30a7027 resource = Database result = Ok(()) uri = /metrics/collectors/6e1e4a25-c6a7-49c5-bb1d-86638130bef7/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmNGM2ZGNlZi0yNTc3LTQxMmUtODNkMS0zNzkyZGNjMDQ2MWQifX0%3D
9672024-12-14T01:15:48.458ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2332 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:57045 req_id = 6edcd001-2663-4a3e-a8fb-1998e30a7027 response_code = 200 uri = /metrics/collectors/6e1e4a25-c6a7-49c5-bb1d-86638130bef7/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmNGM2ZGNlZi0yNTc3LTQxMmUtODNkMS0zNzkyZGNjMDQ2MWQifX0%3D
9682024-12-14T01:15:49.349ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 6c0c03fb-12d4-46b4-b269-f6232cce0a99 roles = RoleSet { roles: {} } uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
9692024-12-14T01:15:49.350ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 6c0c03fb-12d4-46b4-b269-f6232cce0a99 resource = Database result = Ok(()) uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
9702024-12-14T01:15:49.352ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2350 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 6c0c03fb-12d4-46b4-b269-f6232cce0a99 response_code = 200 uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
9712024-12-14T01:15:49.867ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::c]:12221 remote_addr = [fd00:1122:3344:101::2]:53117
9722024-12-14T01:15:49.867ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:53117 req_id = ac1b94ae-e62e-4c2b-97ba-9b7684f313c0 roles = RoleSet { roles: {} } uri = /metrics/producers
9732024-12-14T01:15:49.868ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:53117 req_id = ac1b94ae-e62e-4c2b-97ba-9b7684f313c0 resource = Database result = Ok(()) uri = /metrics/producers
9742024-12-14T01:15:49.904ZINFOnexus: assigned collector to new producer collector_id = 6e1e4a25-c6a7-49c5-bb1d-86638130bef7 file = nexus/src/app/oximeter.rs:76 producer_id = e4fc312c-6d5c-475a-9302-50e5d03bb986
9752024-12-14T01:15:49.904ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 36462 local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:53117 req_id = ac1b94ae-e62e-4c2b-97ba-9b7684f313c0 response_code = 201 uri = /metrics/producers
9762024-12-14T01:15:50.113ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:43741 req_id = cfc842be-a61a-442a-9a71-8a27b8e40dda roles = RoleSet { roles: {} } uri = /nat/ipv4/changeset/10?limit=100
9772024-12-14T01:15:50.113ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:43741 req_id = cfc842be-a61a-442a-9a71-8a27b8e40dda resource = Database result = Ok(()) uri = /nat/ipv4/changeset/10?limit=100
9782024-12-14T01:15:50.116ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 3462 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:43741 req_id = cfc842be-a61a-442a-9a71-8a27b8e40dda response_code = 200 uri = /nat/ipv4/changeset/10?limit=100
9792024-12-14T01:15:50.372ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = a3acc6e5-28e7-4256-a99d-ffad5630b24c roles = RoleSet { roles: {} } uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
9802024-12-14T01:15:50.373ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = a3acc6e5-28e7-4256-a99d-ffad5630b24c resource = Database result = Ok(()) uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
9812024-12-14T01:15:50.374ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1973 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = a3acc6e5-28e7-4256-a99d-ffad5630b24c response_code = 200 uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
9822024-12-14T01:15:50.381ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::c]:12221 remote_addr = [fd00:1122:3344:101::2]:39558
9832024-12-14T01:15:50.381ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:39558 req_id = 63e69717-a02a-4bb7-ab2f-9a62affa80de roles = RoleSet { roles: {} } uri = /metrics/producers
9842024-12-14T01:15:50.382ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:39558 req_id = 63e69717-a02a-4bb7-ab2f-9a62affa80de resource = Database result = Ok(()) uri = /metrics/producers
9852024-12-14T01:15:50.415ZINFOnexus: assigned collector to new producer collector_id = 6e1e4a25-c6a7-49c5-bb1d-86638130bef7 file = nexus/src/app/oximeter.rs:76 producer_id = f4c6dcef-2577-412e-83d1-3792dcc0461d
9862024-12-14T01:15:50.415ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 33496 local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:39558 req_id = 63e69717-a02a-4bb7-ab2f-9a62affa80de response_code = 201 uri = /metrics/producers
9872024-12-14T01:15:51.394ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 4a206252-c922-417f-ad07-eb5b65a7783a roles = RoleSet { roles: {} } uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
9882024-12-14T01:15:51.394ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 4a206252-c922-417f-ad07-eb5b65a7783a resource = Database result = Ok(()) uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
9892024-12-14T01:15:51.396ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2268 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 4a206252-c922-417f-ad07-eb5b65a7783a response_code = 200 uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
9902024-12-14T01:15:52.411ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::c]:12221 remote_addr = [fd00:1122:3344:101::2]:33475
9912024-12-14T01:15:52.411ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:33475 req_id = b04e5534-57fd-4c09-be92-8782ba5401a8 roles = RoleSet { roles: {} } uri = /metrics/producers
9922024-12-14T01:15:52.411ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:33475 req_id = b04e5534-57fd-4c09-be92-8782ba5401a8 resource = Database result = Ok(()) uri = /metrics/producers
9932024-12-14T01:15:52.415ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 0cbbd990-50c0-42d1-9b11-73ac7e87b026 roles = RoleSet { roles: {} } uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
9942024-12-14T01:15:52.415ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 0cbbd990-50c0-42d1-9b11-73ac7e87b026 resource = Database result = Ok(()) uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
9952024-12-14T01:15:52.417ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2148 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 0cbbd990-50c0-42d1-9b11-73ac7e87b026 response_code = 200 uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
9962024-12-14T01:15:52.422ZINFOnexus: assigned collector to new producer collector_id = 6e1e4a25-c6a7-49c5-bb1d-86638130bef7 file = nexus/src/app/oximeter.rs:76 producer_id = 1af89dd0-b706-459f-b45b-199f7c17a5b5
9972024-12-14T01:15:52.422ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 10912 local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:33475 req_id = b04e5534-57fd-4c09-be92-8782ba5401a8 response_code = 201 uri = /metrics/producers
9982024-12-14T01:15:53.436ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 800c4b8a-f871-4fa4-9e3e-ff1b3fed522e roles = RoleSet { roles: {} } uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
9992024-12-14T01:15:53.437ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 800c4b8a-f871-4fa4-9e3e-ff1b3fed522e resource = Database result = Ok(()) uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
10002024-12-14T01:15:53.439ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2575 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 800c4b8a-f871-4fa4-9e3e-ff1b3fed522e response_code = 200 uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
10012024-12-14T01:15:53.461ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1464 local_addr = [fd00:1122:3344:101::c]:54941 method = GET remote_addr = [fd00:1122:3344:101::d]:43526 req_id = ac1e92ef-9e6c-43cc-a5e4-8118ec679a00 response_code = 200 uri = /499bc009-7d23-4916-9be8-0a6a8f4e2f93
10022024-12-14T01:15:54.459ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 1b5344b7-486f-4294-b97e-f9eefa0e242e roles = RoleSet { roles: {} } uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
10032024-12-14T01:15:54.459ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 1b5344b7-486f-4294-b97e-f9eefa0e242e resource = Database result = Ok(()) uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
10042024-12-14T01:15:54.461ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2582 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 1b5344b7-486f-4294-b97e-f9eefa0e242e response_code = 200 uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
10052024-12-14T01:15:54.746ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 26 reason = Timeout
10062024-12-14T01:15:54.746ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 26 reason = Timeout
10072024-12-14T01:15:54.746ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
10082024-12-14T01:15:54.746ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
10092024-12-14T01:15:54.746ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 76 reason = Timeout
10102024-12-14T01:15:54.746ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
10112024-12-14T01:15:54.746ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
10122024-12-14T01:15:54.746ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 28 reason = Timeout
10132024-12-14T01:15:54.746ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
10142024-12-14T01:15:54.746ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 26 reason = Timeout
10152024-12-14T01:15:54.746ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
10162024-12-14T01:15:54.746ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
10172024-12-14T01:15:54.746ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 76 reason = Timeout
10182024-12-14T01:15:54.746ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
10192024-12-14T01:15:54.746ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 26 reason = Timeout
10202024-12-14T01:15:54.746ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 26 reason = Timeout
10212024-12-14T01:15:54.746ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
10222024-12-14T01:15:54.746ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 26 reason = Timeout
10232024-12-14T01:15:54.746ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
10242024-12-14T01:15:54.746ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
10252024-12-14T01:15:54.746ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 26 reason = Timeout
10262024-12-14T01:15:54.746ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
10272024-12-14T01:15:54.746ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 26 reason = Timeout
10282024-12-14T01:15:54.746ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
10292024-12-14T01:15:54.746ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 26 reason = Timeout
10302024-12-14T01:15:54.746ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
10312024-12-14T01:15:54.746ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 26 reason = Timeout
10322024-12-14T01:15:54.746ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 30 reason = Timeout
10332024-12-14T01:15:54.746ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
10342024-12-14T01:15:54.746ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
10352024-12-14T01:15:54.747ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
10362024-12-14T01:15:54.747ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
10372024-12-14T01:15:54.747ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
10382024-12-14T01:15:54.747ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
10392024-12-14T01:15:54.747ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
10402024-12-14T01:15:54.747ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
10412024-12-14T01:15:54.747ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
10422024-12-14T01:15:54.747ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
10432024-12-14T01:15:54.747ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
10442024-12-14T01:15:54.747ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
10452024-12-14T01:15:54.747ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
10462024-12-14T01:15:54.747ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
10472024-12-14T01:15:54.747ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 26 reason = Timeout
10482024-12-14T01:15:54.747ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 27 reason = Timeout
10492024-12-14T01:15:54.747ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 26 reason = Timeout
10502024-12-14T01:15:54.747ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
10512024-12-14T01:15:54.747ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
10522024-12-14T01:15:54.747ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
10532024-12-14T01:15:54.748ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
10542024-12-14T01:15:54.748ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
10552024-12-14T01:15:54.748ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
10562024-12-14T01:15:54.749ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.077519ms iteration = 26 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
10572024-12-14T01:15:54.750ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10582024-12-14T01:15:54.750ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10592024-12-14T01:15:54.751ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-7e423e95-464a-471f-a0d9-4799977f0b0f.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-7e423e95-464a-471f-a0d9-4799977f0b0f.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1517, tv_nsec: 580289048 } })
10602024-12-14T01:15:54.751ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1103
10612024-12-14T01:15:54.751ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10622024-12-14T01:15:54.752ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
10632024-12-14T01:15:54.752ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
10642024-12-14T01:15:54.753ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
10652024-12-14T01:15:54.753ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
10662024-12-14T01:15:54.753ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
10672024-12-14T01:15:54.754ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
10682024-12-14T01:15:54.754ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
10692024-12-14T01:15:54.754ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
10702024-12-14T01:15:54.754ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
10712024-12-14T01:15:54.755ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-7e423e95-464a-471f-a0d9-4799977f0b0f.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-7e423e95-464a-471f-a0d9-4799977f0b0f.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1517, tv_nsec: 583931124 } })
10722024-12-14T01:15:54.755ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1103
10732024-12-14T01:15:54.756ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10742024-12-14T01:15:54.756ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10752024-12-14T01:15:54.759ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
10762024-12-14T01:15:54.759ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
10772024-12-14T01:15:54.759ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
10782024-12-14T01:15:54.759ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
10792024-12-14T01:15:54.760ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
10802024-12-14T01:15:54.760ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
10812024-12-14T01:15:54.760ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 12.543943ms iteration = 26 status = {"errors":[],"records_set_to_done":[]}
10822024-12-14T01:15:54.762ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10832024-12-14T01:15:54.763ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-7e423e95-464a-471f-a0d9-4799977f0b0f.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-7e423e95-464a-471f-a0d9-4799977f0b0f.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1517, tv_nsec: 592058759 } })
10842024-12-14T01:15:54.763ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1103 rack_id = f30fd360-30ae-4088-9821-aa98c2bc8c09
10852024-12-14T01:15:54.763ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
10862024-12-14T01:15:54.763ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
10872024-12-14T01:15:54.764ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
10882024-12-14T01:15:54.764ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
10892024-12-14T01:15:54.764ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
10902024-12-14T01:15:54.765ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10912024-12-14T01:15:54.770ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
10922024-12-14T01:15:54.770ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
10932024-12-14T01:15:54.770ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 8770bfcc-3a71-4738-a6b0-91008cadcb0f, lookup_type: ById(8770bfcc-3a71-4738-a6b0-91008cadcb0f) } result = Ok(())
10942024-12-14T01:15:54.770ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
10952024-12-14T01:15:54.770ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
10962024-12-14T01:15:54.771ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
10972024-12-14T01:15:54.771ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 24.715405ms iteration = 26 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
10982024-12-14T01:15:54.780ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
10992024-12-14T01:15:54.780ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
11002024-12-14T01:15:54.780ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
11012024-12-14T01:15:54.780ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
11022024-12-14T01:15:54.782ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
11032024-12-14T01:15:54.782ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
11042024-12-14T01:15:54.782ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
11052024-12-14T01:15:54.782ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
11062024-12-14T01:15:54.782ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
11072024-12-14T01:15:54.783ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
11082024-12-14T01:15:54.784ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 8770bfcc-3a71-4738-a6b0-91008cadcb0f original_time_created = 2024-12-14 01:03:15.378067 UTC target_id = 8770bfcc-3a71-4738-a6b0-91008cadcb0f time_created = 2024-12-14 01:03:15.378067 UTC
11092024-12-14T01:15:54.784ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 37.697881ms iteration = 76 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"8770bfcc-3a71-4738-a6b0-91008cadcb0f","time_created":"2024-12-14T01:03:15.378067Z"}
11102024-12-14T01:15:54.784ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 37.702181ms iteration = 26 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
11112024-12-14T01:15:54.786ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
11122024-12-14T01:15:54.786ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11132024-12-14T01:15:54.786ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
11142024-12-14T01:15:54.788ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 41.396458ms iteration = 76 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
11152024-12-14T01:15:54.788ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
11162024-12-14T01:15:54.788ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
11172024-12-14T01:15:54.788ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
11182024-12-14T01:15:54.794ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 48.118672ms iteration = 26 status = {"errors":[],"garbage_collect_requested":[]}
11192024-12-14T01:15:54.796ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
11202024-12-14T01:15:54.796ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
11212024-12-14T01:15:54.797ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
11222024-12-14T01:15:54.797ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
11232024-12-14T01:15:54.797ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
11242024-12-14T01:15:54.797ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
11252024-12-14T01:15:54.798ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 51.46382ms iteration = 26 status = {"errors":[],"requests_created_ok":[],"start_invoked_ok":[]}
11262024-12-14T01:15:54.799ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
11272024-12-14T01:15:54.799ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
11282024-12-14T01:15:54.799ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
11292024-12-14T01:15:54.799ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11302024-12-14T01:15:54.799ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
11312024-12-14T01:15:54.800ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11322024-12-14T01:15:54.800ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
11332024-12-14T01:15:54.801ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
11342024-12-14T01:15:54.801ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
11352024-12-14T01:15:54.801ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
11362024-12-14T01:15:54.801ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
11372024-12-14T01:15:54.803ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 7e423e95-464a-471f-a0d9-4799977f0b0f, lookup_type: ById(7e423e95-464a-471f-a0d9-4799977f0b0f) } result = Ok(())
11382024-12-14T01:15:54.803ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
11392024-12-14T01:15:54.803ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
11402024-12-14T01:15:54.806ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 59.785131ms iteration = 26 status = 0
11412024-12-14T01:15:54.811ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
11422024-12-14T01:15:54.811ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
11432024-12-14T01:15:54.813ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11442024-12-14T01:15:54.815ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
11452024-12-14T01:15:54.815ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
11462024-12-14T01:15:54.815ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
11472024-12-14T01:15:54.816ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11482024-12-14T01:15:54.818ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
11492024-12-14T01:15:54.818ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
11502024-12-14T01:15:54.818ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
11512024-12-14T01:15:54.821ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
11522024-12-14T01:15:54.821ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
11532024-12-14T01:15:54.821ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 74.927149ms iteration = 26 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
11542024-12-14T01:15:54.830ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
11552024-12-14T01:15:54.830ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
11562024-12-14T01:15:54.831ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 83.910788ms iteration = 26 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[]}
11572024-12-14T01:15:54.833ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
11582024-12-14T01:15:54.833ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
11592024-12-14T01:15:54.833ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "64c73497-a9cb-482e-a736-977c610fb496", "content-length": "26", "date": "Sat, 14 Dec 2024 01:15:54 GMT"} })
11602024-12-14T01:15:54.833ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
11612024-12-14T01:15:54.836ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
11622024-12-14T01:15:54.840ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 93.443605ms iteration = 30 status = {"physical_disks_added":0}
11632024-12-14T01:15:54.949ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
11642024-12-14T01:15:54.949ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
11652024-12-14T01:15:54.950ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "97a8c587-799a-41fc-ba7f-5ab4b87584ec", "content-length": "26", "date": "Sat, 14 Dec 2024 01:15:54 GMT"} })
11662024-12-14T01:15:54.950ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
11672024-12-14T01:15:54.952ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
11682024-12-14T01:15:54.961ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1111 rack_id = f30fd360-30ae-4088-9821-aa98c2bc8c09 zone_address = fd00:1122:3344:101::2
11692024-12-14T01:15:54.961ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = f30fd360-30ae-4088-9821-aa98c2bc8c09 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
11702024-12-14T01:15:54.962ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = f30fd360-30ae-4088-9821-aa98c2bc8c09 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "262fa0cf-76c8-408e-877b-5407a10a6ff9", "content-length": "26", "date": "Sat, 14 Dec 2024 01:15:54 GMT"} })
11712024-12-14T01:15:54.962ZINFOnexus (BackgroundTasks): VPC route manager sled 7e423e95-464a-471f-a0d9-4799977f0b0f background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
11722024-12-14T01:15:54.962ZDEBGnexus (BackgroundTasks): client request SledAgent = 7e423e95-464a-471f-a0d9-4799977f0b0f background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
11732024-12-14T01:15:54.962ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1116 rack_id = f30fd360-30ae-4088-9821-aa98c2bc8c09 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
11742024-12-14T01:15:54.962ZDEBGnexus (BackgroundTasks): client response SledAgent = 7e423e95-464a-471f-a0d9-4799977f0b0f background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "e929a636-e74a-4a61-a5f3-62fdd0c9f82d", "content-length": "372", "date": "Sat, 14 Dec 2024 01:15:54 GMT"} })
11752024-12-14T01:15:54.962ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
11762024-12-14T01:15:54.963ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
11772024-12-14T01:15:54.964ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = f30fd360-30ae-4088-9821-aa98c2bc8c09
11782024-12-14T01:15:54.981ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 234.705462ms iteration = 26 status = {}
11792024-12-14T01:15:55.011ZDEBGnexus (BackgroundTasks): client request SledAgent = 7e423e95-464a-471f-a0d9-4799977f0b0f background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
11802024-12-14T01:15:55.011ZDEBGnexus (BackgroundTasks): client response SledAgent = 7e423e95-464a-471f-a0d9-4799977f0b0f background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "a8d72390-7e1f-455e-90ac-397f11deb64a", "content-length": "691", "date": "Sat, 14 Dec 2024 01:15:54 GMT"} })
11812024-12-14T01:15:55.011ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = buskin
11822024-12-14T01:15:55.011ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = buskin
11832024-12-14T01:15:55.011ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = buskin
11842024-12-14T01:15:55.013ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 265.534174ms iteration = 27 status = {}
11852024-12-14T01:15:55.024ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = buskin
11862024-12-14T01:15:55.024ZDEBGnexus (BackgroundTasks): client request SledAgent = 7e423e95-464a-471f-a0d9-4799977f0b0f background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
11872024-12-14T01:15:55.024ZDEBGnexus (BackgroundTasks): client response SledAgent = 7e423e95-464a-471f-a0d9-4799977f0b0f background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "b00f6168-984d-4a65-ac71-f32929bfb7fb", "date": "Sat, 14 Dec 2024 01:15:54 GMT"} })
11882024-12-14T01:15:55.024ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
11892024-12-14T01:15:55.024ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
11902024-12-14T01:15:55.027ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
11912024-12-14T01:15:55.027ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
11922024-12-14T01:15:55.031ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
11932024-12-14T01:15:55.031ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
11942024-12-14T01:15:55.034ZINFOnexus (BackgroundTasks): VPC route manager sled 7e423e95-464a-471f-a0d9-4799977f0b0f push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
11952024-12-14T01:15:55.034ZDEBGnexus (BackgroundTasks): client request SledAgent = 7e423e95-464a-471f-a0d9-4799977f0b0f background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
11962024-12-14T01:15:55.034ZDEBGnexus (BackgroundTasks): client response SledAgent = 7e423e95-464a-471f-a0d9-4799977f0b0f background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "b1941728-5363-4337-80f2-bf67b7b61a01", "date": "Sat, 14 Dec 2024 01:15:54 GMT"} })
11972024-12-14T01:15:55.035ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
11982024-12-14T01:15:55.035ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "f76606db-42ce-4a2d-8d7e-bcb3ced4144c", "content-length": "2", "date": "Sat, 14 Dec 2024 01:15:54 GMT"} })
11992024-12-14T01:15:55.037ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 290.401884ms iteration = 28 status = {}
12002024-12-14T01:15:55.037ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
12012024-12-14T01:15:55.037ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
12022024-12-14T01:15:55.039ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 292.798783ms iteration = 26 status = null
12032024-12-14T01:15:55.102ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
12042024-12-14T01:15:55.102ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
12052024-12-14T01:15:55.104ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: bb1a4b50-730e-48ee-a311-8a9b9d7bcb73, rack_id: f30fd360-30ae-4088-9821-aa98c2bc8c09, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(bb9f5a1a-5708-4d58-ba4f-c62102236426) } rack_id = f30fd360-30ae-4088-9821-aa98c2bc8c09 switch_location = Switch0
12062024-12-14T01:15:55.104ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
12072024-12-14T01:15:55.104ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
12082024-12-14T01:15:55.114ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = f30fd360-30ae-4088-9821-aa98c2bc8c09 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
12092024-12-14T01:15:55.114ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = f30fd360-30ae-4088-9821-aa98c2bc8c09 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "aa2e3be2-ea34-41ff-a63a-88db85730e96", "content-length": "134", "date": "Sat, 14 Dec 2024 01:15:54 GMT"} })
12102024-12-14T01:15:55.114ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1746 rack_id = f30fd360-30ae-4088-9821-aa98c2bc8c09 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
12112024-12-14T01:15:55.114ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1773 rack_id = f30fd360-30ae-4088-9821-aa98c2bc8c09 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = bb9f5a1a-5708-4d58-ba4f-c62102236426
12122024-12-14T01:15:55.114ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = f30fd360-30ae-4088-9821-aa98c2bc8c09 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
12132024-12-14T01:15:55.114ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = f30fd360-30ae-4088-9821-aa98c2bc8c09 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
12142024-12-14T01:15:55.114ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = f30fd360-30ae-4088-9821-aa98c2bc8c09 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
12152024-12-14T01:15:55.114ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = f30fd360-30ae-4088-9821-aa98c2bc8c09 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
12162024-12-14T01:15:55.114ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = f30fd360-30ae-4088-9821-aa98c2bc8c09 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
12172024-12-14T01:15:55.114ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1920 rack_id = f30fd360-30ae-4088-9821-aa98c2bc8c09 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
12182024-12-14T01:15:55.115ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = f30fd360-30ae-4088-9821-aa98c2bc8c09 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
12192024-12-14T01:15:55.115ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = f30fd360-30ae-4088-9821-aa98c2bc8c09 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
12202024-12-14T01:15:55.115ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = f30fd360-30ae-4088-9821-aa98c2bc8c09
12212024-12-14T01:15:55.115ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
12222024-12-14T01:15:55.115ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
12232024-12-14T01:15:55.117ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = f30fd360-30ae-4088-9821-aa98c2bc8c09 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
12242024-12-14T01:15:55.117ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = f30fd360-30ae-4088-9821-aa98c2bc8c09 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "e250fd0e-c3c7-420d-8cd6-37299ab33c10", "content-length": "2", "date": "Sat, 14 Dec 2024 01:15:54 GMT"} })
12252024-12-14T01:15:55.117ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = f30fd360-30ae-4088-9821-aa98c2bc8c09 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
12262024-12-14T01:15:55.118ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = f30fd360-30ae-4088-9821-aa98c2bc8c09 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "b36c7b54-62c8-4994-8974-6df4ae301ca4", "content-length": "52", "date": "Sat, 14 Dec 2024 01:15:54 GMT"} })
12272024-12-14T01:15:55.118ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.1.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = f30fd360-30ae-4088-9821-aa98c2bc8c09 switch_location = Switch0
12282024-12-14T01:15:55.118ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = f30fd360-30ae-4088-9821-aa98c2bc8c09 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
12292024-12-14T01:15:55.215ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = f30fd360-30ae-4088-9821-aa98c2bc8c09 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "d83486b8-23c1-4e36-ab2a-f4e7ab27878b", "date": "Sat, 14 Dec 2024 01:15:54 GMT"} })
12302024-12-14T01:15:55.215ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = f30fd360-30ae-4088-9821-aa98c2bc8c09 switch = Switch0
12312024-12-14T01:15:55.215ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = f30fd360-30ae-4088-9821-aa98c2bc8c09 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
12322024-12-14T01:15:55.215ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = f30fd360-30ae-4088-9821-aa98c2bc8c09 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "1ed90ac0-0bb0-4998-8fab-7cf1bc0cc31f", "content-length": "682", "date": "Sat, 14 Dec 2024 01:15:54 GMT"} })
12332024-12-14T01:15:55.216ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
12342024-12-14T01:15:55.216ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
12352024-12-14T01:15:55.220ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
12362024-12-14T01:15:55.220ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
12372024-12-14T01:15:55.224ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
12382024-12-14T01:15:55.224ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
12392024-12-14T01:15:55.225ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
12402024-12-14T01:15:55.226ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
12412024-12-14T01:15:55.234ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 487.477078ms iteration = 26 status = {}
12422024-12-14T01:15:55.256ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::c]:12221 remote_addr = [fd00:1122:3344:101::1]:43582
12432024-12-14T01:15:55.256ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::1]:43582 req_id = b0bbcf41-9c0f-46bf-89e1-b501ef93e00e roles = RoleSet { roles: {} } uri = /metrics/producers
12442024-12-14T01:15:55.257ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::1]:43582 req_id = b0bbcf41-9c0f-46bf-89e1-b501ef93e00e resource = Database result = Ok(()) uri = /metrics/producers
12452024-12-14T01:15:55.296ZINFOnexus: assigned collector to new producer collector_id = 6e1e4a25-c6a7-49c5-bb1d-86638130bef7 file = nexus/src/app/oximeter.rs:76 producer_id = 7e423e95-464a-471f-a0d9-4799977f0b0f
12462024-12-14T01:15:55.296ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 39572 local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::1]:43582 req_id = b0bbcf41-9c0f-46bf-89e1-b501ef93e00e response_code = 201 uri = /metrics/producers
12472024-12-14T01:15:55.480ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 03b8767a-3dac-42f6-a0e8-4ddb37aebf88 roles = RoleSet { roles: {} } uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
12482024-12-14T01:15:55.480ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 03b8767a-3dac-42f6-a0e8-4ddb37aebf88 resource = Database result = Ok(()) uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
12492024-12-14T01:15:55.482ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2076 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 03b8767a-3dac-42f6-a0e8-4ddb37aebf88 response_code = 200 uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
12502024-12-14T01:15:56.503ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 39060f7e-e494-498c-9b5c-307048d76865 roles = RoleSet { roles: {} } uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
12512024-12-14T01:15:56.504ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 39060f7e-e494-498c-9b5c-307048d76865 resource = Database result = Ok(()) uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
12522024-12-14T01:15:56.506ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2476 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 39060f7e-e494-498c-9b5c-307048d76865 response_code = 200 uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
12532024-12-14T01:15:57.526ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 01729345-ff40-49fd-a745-8379aea208bf roles = RoleSet { roles: {} } uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
12542024-12-14T01:15:57.526ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 01729345-ff40-49fd-a745-8379aea208bf resource = Database result = Ok(()) uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
12552024-12-14T01:15:57.529ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 3099 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 01729345-ff40-49fd-a745-8379aea208bf response_code = 200 uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
12562024-12-14T01:15:58.548ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = a0418a74-55e2-4014-82d1-9549dd4a00f9 roles = RoleSet { roles: {} } uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
12572024-12-14T01:15:58.549ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = a0418a74-55e2-4014-82d1-9549dd4a00f9 resource = Database result = Ok(()) uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
12582024-12-14T01:15:58.551ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2447 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = a0418a74-55e2-4014-82d1-9549dd4a00f9 response_code = 200 uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
12592024-12-14T01:15:59.570ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = d1d860a2-c1a5-4b5c-8c34-1867d117f33e roles = RoleSet { roles: {} } uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
12602024-12-14T01:15:59.570ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = d1d860a2-c1a5-4b5c-8c34-1867d117f33e resource = Database result = Ok(()) uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
12612024-12-14T01:15:59.572ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2160 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = d1d860a2-c1a5-4b5c-8c34-1867d117f33e response_code = 200 uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
12622024-12-14T01:16:00.592ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 67c6fa29-ff92-4c2d-a77d-ca19d5c57d69 roles = RoleSet { roles: {} } uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
12632024-12-14T01:16:00.593ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 67c6fa29-ff92-4c2d-a77d-ca19d5c57d69 resource = Database result = Ok(()) uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
12642024-12-14T01:16:00.595ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2551 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 67c6fa29-ff92-4c2d-a77d-ca19d5c57d69 response_code = 200 uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
12652024-12-14T01:16:01.615ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = c7e830da-b572-4ddc-b27d-52fed31f5fed roles = RoleSet { roles: {} } uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
12662024-12-14T01:16:01.615ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = c7e830da-b572-4ddc-b27d-52fed31f5fed resource = Database result = Ok(()) uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
12672024-12-14T01:16:01.617ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2297 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = c7e830da-b572-4ddc-b27d-52fed31f5fed response_code = 200 uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
12682024-12-14T01:16:02.637ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = ab7fb6c1-ebd1-49f3-937a-387bc5e82e23 roles = RoleSet { roles: {} } uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
12692024-12-14T01:16:02.637ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = ab7fb6c1-ebd1-49f3-937a-387bc5e82e23 resource = Database result = Ok(()) uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
12702024-12-14T01:16:02.639ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2448 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = ab7fb6c1-ebd1-49f3-937a-387bc5e82e23 response_code = 200 uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
12712024-12-14T01:16:03.461ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1452 local_addr = [fd00:1122:3344:101::c]:54941 method = GET remote_addr = [fd00:1122:3344:101::d]:43526 req_id = 8d4988d4-d9af-499d-8588-1b8deecb0410 response_code = 200 uri = /499bc009-7d23-4916-9be8-0a6a8f4e2f93
12722024-12-14T01:16:03.659ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 51e1208c-b51d-4fc0-b58c-722aa495203a roles = RoleSet { roles: {} } uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
12732024-12-14T01:16:03.660ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 51e1208c-b51d-4fc0-b58c-722aa495203a resource = Database result = Ok(()) uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
12742024-12-14T01:16:03.660ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::c]:12221 remote_addr = [fd00:1122:3344:101::2]:52837
12752024-12-14T01:16:03.660ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:52837 req_id = 80e3a8ca-061c-4e2d-b48f-f91dec1e2d25 roles = RoleSet { roles: {} } uri = /metrics/producers
12762024-12-14T01:16:03.660ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:52837 req_id = 80e3a8ca-061c-4e2d-b48f-f91dec1e2d25 resource = Database result = Ok(()) uri = /metrics/producers
12772024-12-14T01:16:03.662ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2385 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 51e1208c-b51d-4fc0-b58c-722aa495203a response_code = 200 uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
12782024-12-14T01:16:03.670ZINFOnexus: assigned collector to new producer collector_id = 6e1e4a25-c6a7-49c5-bb1d-86638130bef7 file = nexus/src/app/oximeter.rs:76 producer_id = 4b3b7ff9-1419-46a8-b9c9-d413d48fa5b2
12792024-12-14T01:16:03.670ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 10310 local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:52837 req_id = 80e3a8ca-061c-4e2d-b48f-f91dec1e2d25 response_code = 201 uri = /metrics/producers
12802024-12-14T01:16:04.682ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 4f0c6a80-1168-49cb-a610-2786aca4396b roles = RoleSet { roles: {} } uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
12812024-12-14T01:16:04.683ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 4f0c6a80-1168-49cb-a610-2786aca4396b resource = Database result = Ok(()) uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
12822024-12-14T01:16:04.685ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2440 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 4f0c6a80-1168-49cb-a610-2786aca4396b response_code = 200 uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
12832024-12-14T01:16:04.746ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 77 reason = Timeout
12842024-12-14T01:16:04.746ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 77 reason = Timeout
12852024-12-14T01:16:04.746ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
12862024-12-14T01:16:04.746ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
12872024-12-14T01:16:04.746ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
12882024-12-14T01:16:04.746ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
12892024-12-14T01:16:04.749ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12902024-12-14T01:16:04.751ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
12912024-12-14T01:16:04.751ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
12922024-12-14T01:16:04.751ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
12932024-12-14T01:16:04.754ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
12942024-12-14T01:16:04.754ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
12952024-12-14T01:16:04.756ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12962024-12-14T01:16:04.759ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 8770bfcc-3a71-4738-a6b0-91008cadcb0f, lookup_type: ById(8770bfcc-3a71-4738-a6b0-91008cadcb0f) } result = Ok(())
12972024-12-14T01:16:04.759ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
12982024-12-14T01:16:04.759ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
12992024-12-14T01:16:04.772ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 8770bfcc-3a71-4738-a6b0-91008cadcb0f original_time_created = 2024-12-14 01:03:15.378067 UTC target_id = 8770bfcc-3a71-4738-a6b0-91008cadcb0f time_created = 2024-12-14 01:03:15.378067 UTC
13002024-12-14T01:16:04.772ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 25.886269ms iteration = 77 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"8770bfcc-3a71-4738-a6b0-91008cadcb0f","time_created":"2024-12-14T01:03:15.378067Z"}
13012024-12-14T01:16:04.774ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
13022024-12-14T01:16:04.774ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
13032024-12-14T01:16:04.802ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 56.046882ms iteration = 77 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
13042024-12-14T01:16:05.704ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = b94e1e7d-ac34-4fdd-97af-62e8e1467c73 roles = RoleSet { roles: {} } uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
13052024-12-14T01:16:05.705ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = b94e1e7d-ac34-4fdd-97af-62e8e1467c73 resource = Database result = Ok(()) uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
13062024-12-14T01:16:05.707ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2179 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = b94e1e7d-ac34-4fdd-97af-62e8e1467c73 response_code = 200 uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
13072024-12-14T01:16:06.543ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::c]:12221 remote_addr = [fd00:1122:3344:101::1]:61527
13082024-12-14T01:16:06.543ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::1]:61527 req_id = 0ff95a41-050e-4349-ae34-8ab034d6cffd roles = RoleSet { roles: {} } uri = /metrics/producers
13092024-12-14T01:16:06.544ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::1]:61527 req_id = 0ff95a41-050e-4349-ae34-8ab034d6cffd resource = Database result = Ok(()) uri = /metrics/producers
13102024-12-14T01:16:06.554ZINFOnexus: assigned collector to new producer collector_id = 6e1e4a25-c6a7-49c5-bb1d-86638130bef7 file = nexus/src/app/oximeter.rs:76 producer_id = da183c1c-c285-4d66-ac20-8a6c823e1341
13112024-12-14T01:16:06.554ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 11094 local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::1]:61527 req_id = 0ff95a41-050e-4349-ae34-8ab034d6cffd response_code = 201 uri = /metrics/producers
13122024-12-14T01:16:06.726ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 0a438c47-2a46-4e36-8c8a-c99bc60cd1bf roles = RoleSet { roles: {} } uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
13132024-12-14T01:16:06.727ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 0a438c47-2a46-4e36-8c8a-c99bc60cd1bf resource = Database result = Ok(()) uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
13142024-12-14T01:16:06.728ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1975 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 0a438c47-2a46-4e36-8c8a-c99bc60cd1bf response_code = 200 uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
13152024-12-14T01:16:07.748ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 2dd4bedf-054c-46c0-8147-75f31489b97d roles = RoleSet { roles: {} } uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
13162024-12-14T01:16:07.748ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 2dd4bedf-054c-46c0-8147-75f31489b97d resource = Database result = Ok(()) uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
13172024-12-14T01:16:07.750ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2427 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 2dd4bedf-054c-46c0-8147-75f31489b97d response_code = 200 uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
13182024-12-14T01:16:08.770ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 3b46f234-8e34-486c-956b-96b401896419 roles = RoleSet { roles: {} } uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
13192024-12-14T01:16:08.770ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 3b46f234-8e34-486c-956b-96b401896419 resource = Database result = Ok(()) uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
13202024-12-14T01:16:08.772ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2517 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 3b46f234-8e34-486c-956b-96b401896419 response_code = 200 uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
13212024-12-14T01:16:09.791ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 3766451a-7712-4fbb-b827-86ea6e35609a roles = RoleSet { roles: {} } uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
13222024-12-14T01:16:09.792ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 3766451a-7712-4fbb-b827-86ea6e35609a resource = Database result = Ok(()) uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
13232024-12-14T01:16:09.794ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2503 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 3766451a-7712-4fbb-b827-86ea6e35609a response_code = 200 uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
13242024-12-14T01:16:10.812ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = d8c283cc-0e0f-4318-83a8-7119c774bf10 roles = RoleSet { roles: {} } uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
13252024-12-14T01:16:10.813ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = d8c283cc-0e0f-4318-83a8-7119c774bf10 resource = Database result = Ok(()) uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
13262024-12-14T01:16:10.815ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2547 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = d8c283cc-0e0f-4318-83a8-7119c774bf10 response_code = 200 uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
13272024-12-14T01:16:11.834ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = ad5353bc-2fd0-4a71-96f4-f185ebc1c64b roles = RoleSet { roles: {} } uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
13282024-12-14T01:16:11.835ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = ad5353bc-2fd0-4a71-96f4-f185ebc1c64b resource = Database result = Ok(()) uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
13292024-12-14T01:16:11.836ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2220 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = ad5353bc-2fd0-4a71-96f4-f185ebc1c64b response_code = 200 uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
13302024-12-14T01:16:12.856ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 32a75ee2-14bc-4988-88c7-ad8c31835a34 roles = RoleSet { roles: {} } uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
13312024-12-14T01:16:12.857ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 32a75ee2-14bc-4988-88c7-ad8c31835a34 resource = Database result = Ok(()) uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
13322024-12-14T01:16:12.859ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2642 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 32a75ee2-14bc-4988-88c7-ad8c31835a34 response_code = 200 uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
13332024-12-14T01:16:13.461ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1520 local_addr = [fd00:1122:3344:101::c]:54941 method = GET remote_addr = [fd00:1122:3344:101::d]:43526 req_id = 4e7df924-bd70-452c-8b6e-a4a9f77099a1 response_code = 200 uri = /499bc009-7d23-4916-9be8-0a6a8f4e2f93
13342024-12-14T01:16:13.878ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = cdda5645-2377-42db-bfa5-7a3e6e33a854 roles = RoleSet { roles: {} } uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
13352024-12-14T01:16:13.878ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = cdda5645-2377-42db-bfa5-7a3e6e33a854 resource = Database result = Ok(()) uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
13362024-12-14T01:16:13.880ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2187 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = cdda5645-2377-42db-bfa5-7a3e6e33a854 response_code = 200 uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
13372024-12-14T01:16:14.746ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 78 reason = Timeout
13382024-12-14T01:16:14.746ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 78 reason = Timeout
13392024-12-14T01:16:14.746ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
13402024-12-14T01:16:14.746ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
13412024-12-14T01:16:14.747ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
13422024-12-14T01:16:14.747ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
13432024-12-14T01:16:14.748ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13442024-12-14T01:16:14.749ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
13452024-12-14T01:16:14.749ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
13462024-12-14T01:16:14.751ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 4.865891ms iteration = 78 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
13472024-12-14T01:16:14.751ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
13482024-12-14T01:16:14.751ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
13492024-12-14T01:16:14.751ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
13502024-12-14T01:16:14.753ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
13512024-12-14T01:16:14.754ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
13522024-12-14T01:16:14.755ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13532024-12-14T01:16:14.758ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 8770bfcc-3a71-4738-a6b0-91008cadcb0f, lookup_type: ById(8770bfcc-3a71-4738-a6b0-91008cadcb0f) } result = Ok(())
13542024-12-14T01:16:14.758ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
13552024-12-14T01:16:14.758ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
13562024-12-14T01:16:14.770ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 8770bfcc-3a71-4738-a6b0-91008cadcb0f original_time_created = 2024-12-14 01:03:15.378067 UTC target_id = 8770bfcc-3a71-4738-a6b0-91008cadcb0f time_created = 2024-12-14 01:03:15.378067 UTC
13572024-12-14T01:16:14.770ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 23.861591ms iteration = 78 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"8770bfcc-3a71-4738-a6b0-91008cadcb0f","time_created":"2024-12-14T01:03:15.378067Z"}
13582024-12-14T01:16:14.900ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = d2190c26-30bb-471e-b120-50a97bf70934 roles = RoleSet { roles: {} } uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
13592024-12-14T01:16:14.900ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = d2190c26-30bb-471e-b120-50a97bf70934 resource = Database result = Ok(()) uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
13602024-12-14T01:16:14.902ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2098 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = d2190c26-30bb-471e-b120-50a97bf70934 response_code = 200 uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
13612024-12-14T01:16:15.921ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 292b4363-9410-4407-83aa-30a51460cc92 roles = RoleSet { roles: {} } uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
13622024-12-14T01:16:15.922ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 292b4363-9410-4407-83aa-30a51460cc92 resource = Database result = Ok(()) uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
13632024-12-14T01:16:15.923ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2046 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 292b4363-9410-4407-83aa-30a51460cc92 response_code = 200 uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
13642024-12-14T01:16:16.943ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 8fb24362-6f1b-4eb4-9add-f68a32f3eed9 roles = RoleSet { roles: {} } uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
13652024-12-14T01:16:16.943ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 8fb24362-6f1b-4eb4-9add-f68a32f3eed9 resource = Database result = Ok(()) uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
13662024-12-14T01:16:16.945ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2621 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 8fb24362-6f1b-4eb4-9add-f68a32f3eed9 response_code = 200 uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
13672024-12-14T01:16:17.965ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 8e83199a-c100-455b-baed-a724ed9043bb roles = RoleSet { roles: {} } uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
13682024-12-14T01:16:17.965ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 8e83199a-c100-455b-baed-a724ed9043bb resource = Database result = Ok(()) uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
13692024-12-14T01:16:17.967ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2299 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 8e83199a-c100-455b-baed-a724ed9043bb response_code = 200 uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
13702024-12-14T01:16:18.452ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:57045 req_id = 793466d7-035f-471e-9a6d-39818e58d73f roles = RoleSet { roles: {} } uri = /metrics/collectors/6e1e4a25-c6a7-49c5-bb1d-86638130bef7/producers?sort_by=id_ascending
13712024-12-14T01:16:18.452ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:57045 req_id = 793466d7-035f-471e-9a6d-39818e58d73f resource = Database result = Ok(()) uri = /metrics/collectors/6e1e4a25-c6a7-49c5-bb1d-86638130bef7/producers?sort_by=id_ascending
13722024-12-14T01:16:18.455ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 3273 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:57045 req_id = 793466d7-035f-471e-9a6d-39818e58d73f response_code = 200 uri = /metrics/collectors/6e1e4a25-c6a7-49c5-bb1d-86638130bef7/producers?sort_by=id_ascending
13732024-12-14T01:16:18.455ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:57045 req_id = 9856b44a-1e20-4b6c-9539-cf6cfecda742 roles = RoleSet { roles: {} } uri = /metrics/collectors/6e1e4a25-c6a7-49c5-bb1d-86638130bef7/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmNGM2ZGNlZi0yNTc3LTQxMmUtODNkMS0zNzkyZGNjMDQ2MWQifX0%3D
13742024-12-14T01:16:18.456ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:57045 req_id = 9856b44a-1e20-4b6c-9539-cf6cfecda742 resource = Database result = Ok(()) uri = /metrics/collectors/6e1e4a25-c6a7-49c5-bb1d-86638130bef7/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmNGM2ZGNlZi0yNTc3LTQxMmUtODNkMS0zNzkyZGNjMDQ2MWQifX0%3D
13752024-12-14T01:16:18.457ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1974 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:57045 req_id = 9856b44a-1e20-4b6c-9539-cf6cfecda742 response_code = 200 uri = /metrics/collectors/6e1e4a25-c6a7-49c5-bb1d-86638130bef7/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmNGM2ZGNlZi0yNTc3LTQxMmUtODNkMS0zNzkyZGNjMDQ2MWQifX0%3D
13762024-12-14T01:16:18.987ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 1bcc1db2-a5ff-4a29-9870-1b9284747489 roles = RoleSet { roles: {} } uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
13772024-12-14T01:16:18.987ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 1bcc1db2-a5ff-4a29-9870-1b9284747489 resource = Database result = Ok(()) uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
13782024-12-14T01:16:18.989ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2130 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 1bcc1db2-a5ff-4a29-9870-1b9284747489 response_code = 200 uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
13792024-12-14T01:16:20.008ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 678785ce-e719-4a5f-9a31-1d9937b5ce31 roles = RoleSet { roles: {} } uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
13802024-12-14T01:16:20.009ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 678785ce-e719-4a5f-9a31-1d9937b5ce31 resource = Database result = Ok(()) uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
13812024-12-14T01:16:20.011ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2686 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 678785ce-e719-4a5f-9a31-1d9937b5ce31 response_code = 200 uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
13822024-12-14T01:16:20.145ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:43741 req_id = 32ec5300-56e8-4c19-9459-1a574278c961 roles = RoleSet { roles: {} } uri = /nat/ipv4/changeset/10?limit=100
13832024-12-14T01:16:20.146ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:43741 req_id = 32ec5300-56e8-4c19-9459-1a574278c961 resource = Database result = Ok(()) uri = /nat/ipv4/changeset/10?limit=100
13842024-12-14T01:16:20.149ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 3407 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:43741 req_id = 32ec5300-56e8-4c19-9459-1a574278c961 response_code = 200 uri = /nat/ipv4/changeset/10?limit=100
13852024-12-14T01:16:21.030ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 613387bb-926b-4bdf-9673-3ecee7ac68e5 roles = RoleSet { roles: {} } uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
13862024-12-14T01:16:21.031ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 613387bb-926b-4bdf-9673-3ecee7ac68e5 resource = Database result = Ok(()) uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
13872024-12-14T01:16:21.033ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2734 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 613387bb-926b-4bdf-9673-3ecee7ac68e5 response_code = 200 uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
13882024-12-14T01:16:22.052ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 85578337-8a59-45cd-9f2d-fa2fc81cb709 roles = RoleSet { roles: {} } uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
13892024-12-14T01:16:22.053ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 85578337-8a59-45cd-9f2d-fa2fc81cb709 resource = Database result = Ok(()) uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
13902024-12-14T01:16:22.055ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2324 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 85578337-8a59-45cd-9f2d-fa2fc81cb709 response_code = 200 uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
13912024-12-14T01:16:23.074ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 3f7271c4-0df5-422e-adc4-8bd339d6fcd9 roles = RoleSet { roles: {} } uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
13922024-12-14T01:16:23.075ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 3f7271c4-0df5-422e-adc4-8bd339d6fcd9 resource = Database result = Ok(()) uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
13932024-12-14T01:16:23.076ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1849 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 3f7271c4-0df5-422e-adc4-8bd339d6fcd9 response_code = 200 uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
13942024-12-14T01:16:23.461ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1405 local_addr = [fd00:1122:3344:101::c]:54941 method = GET remote_addr = [fd00:1122:3344:101::d]:43526 req_id = ea26dba4-b206-4292-92e2-f5b2d0bb6517 response_code = 200 uri = /499bc009-7d23-4916-9be8-0a6a8f4e2f93
13952024-12-14T01:16:24.096ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 2b1e3b31-2617-4ab7-8a0a-9d005de9e87d roles = RoleSet { roles: {} } uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
13962024-12-14T01:16:24.097ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 2b1e3b31-2617-4ab7-8a0a-9d005de9e87d resource = Database result = Ok(()) uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
13972024-12-14T01:16:24.098ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2275 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 2b1e3b31-2617-4ab7-8a0a-9d005de9e87d response_code = 200 uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
13982024-12-14T01:16:24.747ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 15 reason = Timeout
13992024-12-14T01:16:24.747ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 29 reason = Timeout
14002024-12-14T01:16:24.747ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
14012024-12-14T01:16:24.747ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
14022024-12-14T01:16:24.747ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 15 reason = Timeout
14032024-12-14T01:16:24.747ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:83 reason = blueprint disabled target_id = 8770bfcc-3a71-4738-a6b0-91008cadcb0f
14042024-12-14T01:16:24.747ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
14052024-12-14T01:16:24.747ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 22.521\u{b5}s iteration = 15 status = {"enabled":false,"target_id":"8770bfcc-3a71-4738-a6b0-91008cadcb0f"}
14062024-12-14T01:16:24.747ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 27 reason = Timeout
14072024-12-14T01:16:24.747ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 27 reason = Timeout
14082024-12-14T01:16:24.747ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
14092024-12-14T01:16:24.747ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
14102024-12-14T01:16:24.747ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 15 reason = Timeout
14112024-12-14T01:16:24.747ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 31 reason = Timeout
14122024-12-14T01:16:24.747ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
14132024-12-14T01:16:24.747ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 27 reason = Timeout
14142024-12-14T01:16:24.747ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
14152024-12-14T01:16:24.747ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 14 reason = Timeout
14162024-12-14T01:16:24.747ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
14172024-12-14T01:16:24.747ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 27 reason = Timeout
14182024-12-14T01:16:24.747ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
14192024-12-14T01:16:24.747ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 27 reason = Timeout
14202024-12-14T01:16:24.747ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 14 reason = Timeout
14212024-12-14T01:16:24.747ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
14222024-12-14T01:16:24.747ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2024-12-14 01:06:24.747279522 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
14232024-12-14T01:16:24.747ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
14242024-12-14T01:16:24.747ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
14252024-12-14T01:16:24.747ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 27 reason = Timeout
14262024-12-14T01:16:24.747ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
14272024-12-14T01:16:24.747ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 27 reason = Timeout
14282024-12-14T01:16:24.747ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
14292024-12-14T01:16:24.747ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 27 reason = Timeout
14302024-12-14T01:16:24.747ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
14312024-12-14T01:16:24.747ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 16 reason = Timeout
14322024-12-14T01:16:24.747ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 27 reason = Timeout
14332024-12-14T01:16:24.747ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
14342024-12-14T01:16:24.747ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 28 reason = Timeout
14352024-12-14T01:16:24.747ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
14362024-12-14T01:16:24.747ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 16 reason = Timeout
14372024-12-14T01:16:24.747ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 14 reason = Timeout
14382024-12-14T01:16:24.747ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:56
14392024-12-14T01:16:24.747ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 15 reason = Timeout
14402024-12-14T01:16:24.747ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
14412024-12-14T01:16:24.747ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 79 reason = Timeout
14422024-12-14T01:16:24.747ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
14432024-12-14T01:16:24.747ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 27 reason = Timeout
14442024-12-14T01:16:24.747ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 15 reason = Timeout
14452024-12-14T01:16:24.747ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
14462024-12-14T01:16:24.747ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 79 reason = Timeout
14472024-12-14T01:16:24.747ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
14482024-12-14T01:16:24.747ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
14492024-12-14T01:16:24.747ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 14 reason = Timeout
14502024-12-14T01:16:24.747ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 27 reason = Timeout
14512024-12-14T01:16:24.747ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
14522024-12-14T01:16:24.747ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
14532024-12-14T01:16:24.747ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 27 reason = Timeout
14542024-12-14T01:16:24.747ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 15 reason = Timeout
14552024-12-14T01:16:24.747ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
14562024-12-14T01:16:24.747ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
14572024-12-14T01:16:24.747ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
14582024-12-14T01:16:24.747ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 14 reason = Timeout
14592024-12-14T01:16:24.747ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
14602024-12-14T01:16:24.747ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
14612024-12-14T01:16:24.747ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
14622024-12-14T01:16:24.747ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
14632024-12-14T01:16:24.747ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
14642024-12-14T01:16:24.747ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
14652024-12-14T01:16:24.747ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
14662024-12-14T01:16:24.747ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
14672024-12-14T01:16:24.747ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
14682024-12-14T01:16:24.747ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
14692024-12-14T01:16:24.747ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
14702024-12-14T01:16:24.747ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
14712024-12-14T01:16:24.748ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
14722024-12-14T01:16:24.748ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
14732024-12-14T01:16:24.748ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
14742024-12-14T01:16:24.748ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
14752024-12-14T01:16:24.748ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
14762024-12-14T01:16:24.748ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
14772024-12-14T01:16:24.748ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
14782024-12-14T01:16:24.748ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
14792024-12-14T01:16:24.748ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
14802024-12-14T01:16:24.748ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("381b8112-f609-4c4d-ac80-0170ad8fc4bd.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("795aee12-3394-49fa-9cb5-487c97a85b2d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("795aee12-3394-49fa-9cb5-487c97a85b2d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::9))) }], valid_until: Instant { tv_sec: 1547, tv_nsec: 577112828 } })
14812024-12-14T01:16:24.748ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
14822024-12-14T01:16:24.748ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
14832024-12-14T01:16:24.748ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("1b585a7c-7eb0-4894-b8b3-3ef1983891e5.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("36100ee5-fa9a-4857-a844-70bec0e8601f.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("830b88b6-26da-4988-83b1-cf4193531cd5.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("830b88b6-26da-4988-83b1-cf4193531cd5.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:2::1))) }], valid_until: Instant { tv_sec: 1547, tv_nsec: 577143039 } })
14842024-12-14T01:16:24.748ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
14852024-12-14T01:16:24.748ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
14862024-12-14T01:16:24.748ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } dns_group = external
14872024-12-14T01:16:24.748ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.395271ms iteration = 15 status = {"addresses":["[fd00:1122:3344:101::8]:5353","[fd00:1122:3344:101::9]:5353"]}
14882024-12-14T01:16:24.748ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } dns_group = internal
14892024-12-14T01:16:24.748ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.136303ms iteration = 15 status = {"addresses":["[fd00:1122:3344:1::1]:5353","[fd00:1122:3344:3::1]:5353","[fd00:1122:3344:2::1]:5353"]}
14902024-12-14T01:16:24.750ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14912024-12-14T01:16:24.750ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 3.567084ms iteration = 27 status = {"errors":[],"garbage_collect_requested":[]}
14922024-12-14T01:16:24.751ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14932024-12-14T01:16:24.751ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 4.052487ms iteration = 27 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
14942024-12-14T01:16:24.752ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 4.894332ms iteration = 14 status = {"errors":[],"found_port_ok":[]}
14952024-12-14T01:16:24.752ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14962024-12-14T01:16:24.752ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
14972024-12-14T01:16:24.752ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
14982024-12-14T01:16:24.752ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
14992024-12-14T01:16:24.753ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
15002024-12-14T01:16:24.753ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
15012024-12-14T01:16:24.753ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15022024-12-14T01:16:24.753ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
15032024-12-14T01:16:24.754ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
15042024-12-14T01:16:24.754ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
15052024-12-14T01:16:24.754ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
15062024-12-14T01:16:24.756ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15072024-12-14T01:16:24.756ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-7e423e95-464a-471f-a0d9-4799977f0b0f.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-7e423e95-464a-471f-a0d9-4799977f0b0f.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1547, tv_nsec: 585433559 } })
15082024-12-14T01:16:24.756ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1103
15092024-12-14T01:16:24.756ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
15102024-12-14T01:16:24.756ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
15112024-12-14T01:16:24.756ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
15122024-12-14T01:16:24.757ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
15132024-12-14T01:16:24.757ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
15142024-12-14T01:16:24.758ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
15152024-12-14T01:16:24.758ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-7e423e95-464a-471f-a0d9-4799977f0b0f.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-7e423e95-464a-471f-a0d9-4799977f0b0f.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1547, tv_nsec: 587424067 } })
15162024-12-14T01:16:24.758ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1103
15172024-12-14T01:16:24.759ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2024-12-14 01:06:24.747279522 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
15182024-12-14T01:16:24.759ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 12.115771ms iteration = 14 status = {"expiration":"2024-12-14T01:06:24.747279522Z","pruned":[]}
15192024-12-14T01:16:24.759ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
15202024-12-14T01:16:24.759ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15212024-12-14T01:16:24.759ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
15222024-12-14T01:16:24.762ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
15232024-12-14T01:16:24.762ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
15242024-12-14T01:16:24.762ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
15252024-12-14T01:16:24.763ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
15262024-12-14T01:16:24.763ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
15272024-12-14T01:16:24.765ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15282024-12-14T01:16:24.766ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15292024-12-14T01:16:24.766ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
15302024-12-14T01:16:24.767ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
15312024-12-14T01:16:24.767ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
15322024-12-14T01:16:24.767ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
15332024-12-14T01:16:24.767ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
15342024-12-14T01:16:24.767ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
15352024-12-14T01:16:24.767ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
15362024-12-14T01:16:24.768ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
15372024-12-14T01:16:24.768ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
15382024-12-14T01:16:24.768ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
15392024-12-14T01:16:24.769ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
15402024-12-14T01:16:24.769ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
15412024-12-14T01:16:24.769ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
15422024-12-14T01:16:24.769ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
15432024-12-14T01:16:24.772ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15442024-12-14T01:16:24.772ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15452024-12-14T01:16:24.775ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
15462024-12-14T01:16:24.777ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
15472024-12-14T01:16:24.777ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 29.977158ms iteration = 27 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
15482024-12-14T01:16:24.777ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
15492024-12-14T01:16:24.779ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15502024-12-14T01:16:24.780ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
15512024-12-14T01:16:24.780ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
15522024-12-14T01:16:24.781ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
15532024-12-14T01:16:24.781ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
15542024-12-14T01:16:24.782ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
15552024-12-14T01:16:24.782ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
15562024-12-14T01:16:24.782ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
15572024-12-14T01:16:24.784ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
15582024-12-14T01:16:24.784ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
15592024-12-14T01:16:24.784ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
15602024-12-14T01:16:24.785ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
15612024-12-14T01:16:24.787ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
15622024-12-14T01:16:24.787ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
15632024-12-14T01:16:24.795ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
15642024-12-14T01:16:24.795ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
15652024-12-14T01:16:24.799ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 52.260192ms iteration = 27 status = {"errors":[],"records_set_to_done":[]}
15662024-12-14T01:16:24.803ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
15672024-12-14T01:16:24.803ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
15682024-12-14T01:16:24.803ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
15692024-12-14T01:16:24.803ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:171 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
15702024-12-14T01:16:24.803ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 56.268748ms iteration = 14 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
15712024-12-14T01:16:24.805ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15722024-12-14T01:16:24.807ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
15732024-12-14T01:16:24.807ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
15742024-12-14T01:16:24.807ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
15752024-12-14T01:16:24.807ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
15762024-12-14T01:16:24.807ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
15772024-12-14T01:16:24.808ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
15782024-12-14T01:16:24.808ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
15792024-12-14T01:16:24.808ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
15802024-12-14T01:16:24.808ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
15812024-12-14T01:16:24.808ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
15822024-12-14T01:16:24.808ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15832024-12-14T01:16:24.810ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
15842024-12-14T01:16:24.810ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:140 found = 0 not_ready_to_be_deleted = 0
15852024-12-14T01:16:24.810ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 62.678634ms iteration = 14 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
15862024-12-14T01:16:24.811ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
15872024-12-14T01:16:24.811ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
15882024-12-14T01:16:24.811ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
15892024-12-14T01:16:24.811ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
15902024-12-14T01:16:24.811ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
15912024-12-14T01:16:24.811ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
15922024-12-14T01:16:24.811ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
15932024-12-14T01:16:24.812ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
15942024-12-14T01:16:24.834ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
15952024-12-14T01:16:24.834ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
15962024-12-14T01:16:24.869ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
15972024-12-14T01:16:24.870ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
15982024-12-14T01:16:24.870ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
15992024-12-14T01:16:24.871ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "a0fdabce-63c4-4440-b025-f736b4626ade", "content-length": "26", "date": "Sat, 14 Dec 2024 01:16:24 GMT"} })
16002024-12-14T01:16:24.871ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
16012024-12-14T01:16:24.873ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
16022024-12-14T01:16:24.881ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
16032024-12-14T01:16:24.881ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
16042024-12-14T01:16:24.881ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
16052024-12-14T01:16:24.895ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
16062024-12-14T01:16:24.895ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
16072024-12-14T01:16:24.896ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "f5039102-3e3c-41b2-bbad-0dc53ea35b78", "content-length": "26", "date": "Sat, 14 Dec 2024 01:16:24 GMT"} })
16082024-12-14T01:16:24.896ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
16092024-12-14T01:16:24.898ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
16102024-12-14T01:16:24.932ZINFOnexus (BackgroundTasks): VPC route manager sled 7e423e95-464a-471f-a0d9-4799977f0b0f background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
16112024-12-14T01:16:24.932ZDEBGnexus (BackgroundTasks): client request SledAgent = 7e423e95-464a-471f-a0d9-4799977f0b0f background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
16122024-12-14T01:16:24.933ZDEBGnexus (BackgroundTasks): client response SledAgent = 7e423e95-464a-471f-a0d9-4799977f0b0f background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "02282932-6a7c-46f2-9390-175a40ec4287", "content-length": "372", "date": "Sat, 14 Dec 2024 01:16:24 GMT"} })
16132024-12-14T01:16:24.933ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
16142024-12-14T01:16:24.933ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
16152024-12-14T01:16:24.934ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16162024-12-14T01:16:24.934ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 187.65544ms iteration = 27 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
16172024-12-14T01:16:24.937ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 8770bfcc-3a71-4738-a6b0-91008cadcb0f, lookup_type: ById(8770bfcc-3a71-4738-a6b0-91008cadcb0f) } result = Ok(())
16182024-12-14T01:16:24.937ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
16192024-12-14T01:16:24.937ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
16202024-12-14T01:16:24.939ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 191.992496ms iteration = 79 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
16212024-12-14T01:16:24.940ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
16222024-12-14T01:16:24.941ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
16232024-12-14T01:16:24.941ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16242024-12-14T01:16:24.942ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-7e423e95-464a-471f-a0d9-4799977f0b0f.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-7e423e95-464a-471f-a0d9-4799977f0b0f.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1547, tv_nsec: 771086322 } })
16252024-12-14T01:16:24.942ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1103 rack_id = f30fd360-30ae-4088-9821-aa98c2bc8c09
16262024-12-14T01:16:24.945ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
16272024-12-14T01:16:24.945ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
16282024-12-14T01:16:24.945ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
16292024-12-14T01:16:24.957ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 8770bfcc-3a71-4738-a6b0-91008cadcb0f original_time_created = 2024-12-14 01:03:15.378067 UTC target_id = 8770bfcc-3a71-4738-a6b0-91008cadcb0f time_created = 2024-12-14 01:03:15.378067 UTC
16302024-12-14T01:16:24.957ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 210.226964ms iteration = 79 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"8770bfcc-3a71-4738-a6b0-91008cadcb0f","time_created":"2024-12-14T01:03:15.378067Z"}
16312024-12-14T01:16:24.980ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
16322024-12-14T01:16:24.980ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
16332024-12-14T01:16:24.980ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
16342024-12-14T01:16:24.980ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
16352024-12-14T01:16:24.984ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
16362024-12-14T01:16:24.985ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
16372024-12-14T01:16:24.989ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16382024-12-14T01:16:24.990ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
16392024-12-14T01:16:24.990ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
16402024-12-14T01:16:24.990ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 243.273989ms iteration = 27 status = {"errors":[],"requests_created_ok":[],"start_invoked_ok":[]}
16412024-12-14T01:16:24.993ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
16422024-12-14T01:16:24.993ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = d9c36d0c-d7d2-439f-a748-798b6d8dea0c dns_zone_name = oxide.test found_so_far = 0 version = 2
16432024-12-14T01:16:24.993ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
16442024-12-14T01:16:24.993ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
16452024-12-14T01:16:24.993ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
16462024-12-14T01:16:24.993ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
16472024-12-14T01:16:24.993ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
16482024-12-14T01:16:24.997ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16492024-12-14T01:16:24.998ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16502024-12-14T01:16:25.001ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
16512024-12-14T01:16:25.001ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
16522024-12-14T01:16:25.001ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 7e423e95-464a-471f-a0d9-4799977f0b0f, lookup_type: ById(7e423e95-464a-471f-a0d9-4799977f0b0f) } result = Ok(())
16532024-12-14T01:16:25.001ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
16542024-12-14T01:16:25.001ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
16552024-12-14T01:16:25.001ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
16562024-12-14T01:16:25.003ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 256.640237ms iteration = 27 status = 0
16572024-12-14T01:16:25.004ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
16582024-12-14T01:16:25.009ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = buskin
16592024-12-14T01:16:25.009ZDEBGnexus (BackgroundTasks): client request SledAgent = 7e423e95-464a-471f-a0d9-4799977f0b0f background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
16602024-12-14T01:16:25.009ZDEBGnexus (BackgroundTasks): client response SledAgent = 7e423e95-464a-471f-a0d9-4799977f0b0f background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "6c177de5-63e0-4f0f-b6d6-953fb18a04b8", "date": "Sat, 14 Dec 2024 01:16:24 GMT"} })
16612024-12-14T01:16:25.009ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
16622024-12-14T01:16:25.010ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
16632024-12-14T01:16:25.013ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
16642024-12-14T01:16:25.013ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
16652024-12-14T01:16:25.013ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
16662024-12-14T01:16:25.013ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 265.901225ms iteration = 27 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
16672024-12-14T01:16:25.013ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
16682024-12-14T01:16:25.018ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
16692024-12-14T01:16:25.018ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
16702024-12-14T01:16:25.021ZINFOnexus (BackgroundTasks): VPC route manager sled 7e423e95-464a-471f-a0d9-4799977f0b0f push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
16712024-12-14T01:16:25.021ZDEBGnexus (BackgroundTasks): client request SledAgent = 7e423e95-464a-471f-a0d9-4799977f0b0f background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
16722024-12-14T01:16:25.022ZDEBGnexus (BackgroundTasks): client response SledAgent = 7e423e95-464a-471f-a0d9-4799977f0b0f background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "820c3d79-22c2-4e7e-b33a-9996e21e0048", "date": "Sat, 14 Dec 2024 01:16:24 GMT"} })
16732024-12-14T01:16:25.023ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 276.822071ms iteration = 29 status = {}
16742024-12-14T01:16:25.024ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 277.238733ms iteration = 27 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[]}
16752024-12-14T01:16:25.031ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16762024-12-14T01:16:25.033ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16772024-12-14T01:16:25.033ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
16782024-12-14T01:16:25.033ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
16792024-12-14T01:16:25.034ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
16802024-12-14T01:16:25.034ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16812024-12-14T01:16:25.035ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
16822024-12-14T01:16:25.035ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 866c9448-c447-4b92-96b6-611d51967e09 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
16832024-12-14T01:16:25.035ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
16842024-12-14T01:16:25.036ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
16852024-12-14T01:16:25.037ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
16862024-12-14T01:16:25.037ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
16872024-12-14T01:16:25.037ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
16882024-12-14T01:16:25.037ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
16892024-12-14T01:16:25.037ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
16902024-12-14T01:16:25.037ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
16912024-12-14T01:16:25.040ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = d9c36d0c-d7d2-439f-a748-798b6d8dea0c dns_zone_name = oxide.test found_so_far = 1 version = 2
16922024-12-14T01:16:25.040ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
16932024-12-14T01:16:25.040ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2024-12-14 01:03:53.017536 UTC dns_group = external generation = 2
16942024-12-14T01:16:25.040ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 292.535466ms iteration = 15 status = {"generation":2}
16952024-12-14T01:16:25.051ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 304.599215ms iteration = 31 status = {"physical_disks_added":0}
16962024-12-14T01:16:25.052ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
16972024-12-14T01:16:25.053ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "eeac9e35-805f-4838-b6cc-be1f1291dacf", "date": "Sat, 14 Dec 2024 01:16:25 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
16982024-12-14T01:16:25.053ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
16992024-12-14T01:16:25.055ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "3f025a1d-3242-4d64-93f3-9aaefe3687c1", "date": "Sat, 14 Dec 2024 01:16:25 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
17002024-12-14T01:16:25.055ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
17012024-12-14T01:16:25.057ZWARNnexus (BackgroundTasks): instance reincarnation completed with errors background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:103 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0 query_errors = 0 restart_errors = 0
17022024-12-14T01:16:25.057ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 309.671231ms iteration = 14 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
17032024-12-14T01:16:25.057ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
17042024-12-14T01:16:25.057ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 309.95721ms iteration = 16 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
17052024-12-14T01:16:25.066ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17062024-12-14T01:16:25.068ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
17072024-12-14T01:16:25.068ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
17082024-12-14T01:16:25.069ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
17092024-12-14T01:16:25.069ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2024-12-14T01:03:19.260831Z, time_modified: 2024-12-14T01:03:19.260831Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 2c668072-a9a1-4a69-928c-e118c99521be, db_silo: Silo { identity: SiloIdentity { id: 2c668072-a9a1-4a69-928c-e118c99521be, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-12-14T01:03:52.950638Z, time_modified: 2024-12-14T01:03:52.950638Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "ad8fd43e0fc4f6284dc5e0b0d0f2e6a6e8d979d13f86a71c9b81d6817e439efd" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 2c668072-a9a1-4a69-928c-e118c99521be, db_silo: Silo { identity: SiloIdentity { id: 2c668072-a9a1-4a69-928c-e118c99521be, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-12-14T01:03:52.950638Z, time_modified: 2024-12-14T01:03:52.950638Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "ad8fd43e0fc4f6284dc5e0b0d0f2e6a6e8d979d13f86a71c9b81d6817e439efd" }] }) }
17102024-12-14T01:16:25.069ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 322.501853ms iteration = 15 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"2c668072-a9a1-4a69-928c-e118c99521be","tls_certs":["ad8fd43e0fc4f6284dc5e0b0d0f2e6a6e8d979d13f86a71c9b81d6817e439efd"]}},"default_endpoint":{"silo_id":"2c668072-a9a1-4a69-928c-e118c99521be","tls_certs":["ad8fd43e0fc4f6284dc5e0b0d0f2e6a6e8d979d13f86a71c9b81d6817e439efd"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
17112024-12-14T01:16:25.070ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1111 rack_id = f30fd360-30ae-4088-9821-aa98c2bc8c09 zone_address = fd00:1122:3344:101::2
17122024-12-14T01:16:25.070ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = f30fd360-30ae-4088-9821-aa98c2bc8c09 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
17132024-12-14T01:16:25.071ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = f30fd360-30ae-4088-9821-aa98c2bc8c09 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "62be779f-e6ca-4d8e-8c6b-9b646f3cfa0d", "content-length": "26", "date": "Sat, 14 Dec 2024 01:16:24 GMT"} })
17142024-12-14T01:16:25.071ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1116 rack_id = f30fd360-30ae-4088-9821-aa98c2bc8c09 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
17152024-12-14T01:16:25.071ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 866c9448-c447-4b92-96b6-611d51967e09 dns_zone_name = control-plane.oxide.internal found_so_far = 51 version = 1
17162024-12-14T01:16:25.071ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
17172024-12-14T01:16:25.071ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2024-12-14 01:03:51.993091 UTC dns_group = internal generation = 1
17182024-12-14T01:16:25.071ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 324.289325ms iteration = 15 status = {"generation":1}
17192024-12-14T01:16:25.073ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = f30fd360-30ae-4088-9821-aa98c2bc8c09
17202024-12-14T01:16:25.118ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = d100cf5a-8198-419d-8580-108cf1b413e8 roles = RoleSet { roles: {} } uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
17212024-12-14T01:16:25.118ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = d100cf5a-8198-419d-8580-108cf1b413e8 resource = Database result = Ok(()) uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
17222024-12-14T01:16:25.120ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2296 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = d100cf5a-8198-419d-8580-108cf1b413e8 response_code = 200 uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
17232024-12-14T01:16:25.133ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
17242024-12-14T01:16:25.134ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "3bb6473b-db1c-4033-a0a1-590a4fb8b295", "date": "Sat, 14 Dec 2024 01:16:25 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
17252024-12-14T01:16:25.134ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
17262024-12-14T01:16:25.136ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "7867ee7a-539c-49a4-9b86-30675576e214", "date": "Sat, 14 Dec 2024 01:16:25 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
17272024-12-14T01:16:25.136ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
17282024-12-14T01:16:25.138ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "cf20d9bd-dfd7-40ce-a073-4d51bec384c4", "date": "Sat, 14 Dec 2024 01:16:25 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
17292024-12-14T01:16:25.138ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
17302024-12-14T01:16:25.140ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
17312024-12-14T01:16:25.140ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 393.21412ms iteration = 16 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
17322024-12-14T01:16:25.146ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
17332024-12-14T01:16:25.146ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "babe0082-c805-4969-a910-c49b832acd73", "content-length": "2", "date": "Sat, 14 Dec 2024 01:16:25 GMT"} })
17342024-12-14T01:16:25.149ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
17352024-12-14T01:16:25.149ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
17362024-12-14T01:16:25.151ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 404.163666ms iteration = 27 status = null
17372024-12-14T01:16:25.155ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 408.465841ms iteration = 27 status = {}
17382024-12-14T01:16:25.172ZDEBGnexus (BackgroundTasks): client request SledAgent = 7e423e95-464a-471f-a0d9-4799977f0b0f background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
17392024-12-14T01:16:25.172ZDEBGnexus (BackgroundTasks): client response SledAgent = 7e423e95-464a-471f-a0d9-4799977f0b0f background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "54c29b2d-ec73-4038-ab80-d307aebaee3d", "content-length": "691", "date": "Sat, 14 Dec 2024 01:16:24 GMT"} })
17402024-12-14T01:16:25.172ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = buskin
17412024-12-14T01:16:25.172ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = buskin
17422024-12-14T01:16:25.172ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = buskin
17432024-12-14T01:16:25.174ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 427.44061ms iteration = 28 status = {}
17442024-12-14T01:16:25.229ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
17452024-12-14T01:16:25.230ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
17462024-12-14T01:16:25.232ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: bb1a4b50-730e-48ee-a311-8a9b9d7bcb73, rack_id: f30fd360-30ae-4088-9821-aa98c2bc8c09, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(bb9f5a1a-5708-4d58-ba4f-c62102236426) } rack_id = f30fd360-30ae-4088-9821-aa98c2bc8c09 switch_location = Switch0
17472024-12-14T01:16:25.232ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
17482024-12-14T01:16:25.232ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
17492024-12-14T01:16:25.243ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = f30fd360-30ae-4088-9821-aa98c2bc8c09 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
17502024-12-14T01:16:25.243ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = f30fd360-30ae-4088-9821-aa98c2bc8c09 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "b4bbe3b8-3b73-4ee0-bcc0-cfda0fbad267", "content-length": "134", "date": "Sat, 14 Dec 2024 01:16:25 GMT"} })
17512024-12-14T01:16:25.243ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1746 rack_id = f30fd360-30ae-4088-9821-aa98c2bc8c09 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
17522024-12-14T01:16:25.243ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1773 rack_id = f30fd360-30ae-4088-9821-aa98c2bc8c09 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = bb9f5a1a-5708-4d58-ba4f-c62102236426
17532024-12-14T01:16:25.243ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = f30fd360-30ae-4088-9821-aa98c2bc8c09 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
17542024-12-14T01:16:25.243ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = f30fd360-30ae-4088-9821-aa98c2bc8c09 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
17552024-12-14T01:16:25.243ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = f30fd360-30ae-4088-9821-aa98c2bc8c09 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
17562024-12-14T01:16:25.243ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = f30fd360-30ae-4088-9821-aa98c2bc8c09 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
17572024-12-14T01:16:25.243ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = f30fd360-30ae-4088-9821-aa98c2bc8c09 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
17582024-12-14T01:16:25.243ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1920 rack_id = f30fd360-30ae-4088-9821-aa98c2bc8c09 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
17592024-12-14T01:16:25.243ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = f30fd360-30ae-4088-9821-aa98c2bc8c09 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
17602024-12-14T01:16:25.243ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = f30fd360-30ae-4088-9821-aa98c2bc8c09 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
17612024-12-14T01:16:25.244ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = f30fd360-30ae-4088-9821-aa98c2bc8c09
17622024-12-14T01:16:25.244ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
17632024-12-14T01:16:25.244ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
17642024-12-14T01:16:25.246ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = f30fd360-30ae-4088-9821-aa98c2bc8c09 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
17652024-12-14T01:16:25.246ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = f30fd360-30ae-4088-9821-aa98c2bc8c09 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "310ae66b-a75a-40db-a684-e531fade3cab", "content-length": "2", "date": "Sat, 14 Dec 2024 01:16:25 GMT"} })
17662024-12-14T01:16:25.246ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = f30fd360-30ae-4088-9821-aa98c2bc8c09 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
17672024-12-14T01:16:25.246ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = f30fd360-30ae-4088-9821-aa98c2bc8c09 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "529ba058-95d7-4b23-8498-37c5b386ac43", "content-length": "52", "date": "Sat, 14 Dec 2024 01:16:25 GMT"} })
17682024-12-14T01:16:25.246ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.1.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = f30fd360-30ae-4088-9821-aa98c2bc8c09 switch_location = Switch0
17692024-12-14T01:16:25.246ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = f30fd360-30ae-4088-9821-aa98c2bc8c09 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
17702024-12-14T01:16:25.343ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = f30fd360-30ae-4088-9821-aa98c2bc8c09 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "f69028f9-49ba-434b-8a26-35540a605020", "date": "Sat, 14 Dec 2024 01:16:24 GMT"} })
17712024-12-14T01:16:25.343ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = f30fd360-30ae-4088-9821-aa98c2bc8c09 switch = Switch0
17722024-12-14T01:16:25.343ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = f30fd360-30ae-4088-9821-aa98c2bc8c09 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
17732024-12-14T01:16:25.343ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = f30fd360-30ae-4088-9821-aa98c2bc8c09 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "23de5bf1-16e0-4b08-9289-56edafd70ae9", "content-length": "682", "date": "Sat, 14 Dec 2024 01:16:24 GMT"} })
17742024-12-14T01:16:25.343ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
17752024-12-14T01:16:25.344ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
17762024-12-14T01:16:25.348ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
17772024-12-14T01:16:25.348ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
17782024-12-14T01:16:25.351ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
17792024-12-14T01:16:25.352ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
17802024-12-14T01:16:25.354ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
17812024-12-14T01:16:25.354ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
17822024-12-14T01:16:25.361ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 613.494724ms iteration = 27 status = {}
17832024-12-14T01:16:25.711ZDEBGnexus (producer-registration-task): registering / renewing oximeter producer lease with Nexus
17842024-12-14T01:16:25.711ZDEBGnexus (producer-registration-task): using nexus address for registration addr = [fd00:1122:3344:101::c]:12221
17852024-12-14T01:16:25.759ZDEBGnexus (producer-registration-task): client request body = Some(Body) method = POST uri = http://[fd00:1122:3344:101::c]:12221/metrics/producers
17862024-12-14T01:16:25.759ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::c]:12221 remote_addr = [fd00:1122:3344:101::c]:42729
17872024-12-14T01:16:25.759ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:42729 req_id = 53ebbb36-8c33-406e-8f8f-b1ca86375050 roles = RoleSet { roles: {} } uri = /metrics/producers
17882024-12-14T01:16:25.759ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:42729 req_id = 53ebbb36-8c33-406e-8f8f-b1ca86375050 resource = Database result = Ok(()) uri = /metrics/producers
17892024-12-14T01:16:25.804ZINFOnexus: assigned collector to new producer collector_id = 6e1e4a25-c6a7-49c5-bb1d-86638130bef7 file = nexus/src/app/oximeter.rs:76 producer_id = 499bc009-7d23-4916-9be8-0a6a8f4e2f93
17902024-12-14T01:16:25.804ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 45045 local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:42729 req_id = 53ebbb36-8c33-406e-8f8f-b1ca86375050 response_code = 201 uri = /metrics/producers
17912024-12-14T01:16:25.804ZDEBGnexus (producer-registration-task): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::c]:12221/metrics/producers", status: 201, headers: {"content-type": "application/json", "x-request-id": "53ebbb36-8c33-406e-8f8f-b1ca86375050", "content-length": "41", "date": "Sat, 14 Dec 2024 01:16:25 GMT"} })
17922024-12-14T01:16:25.806ZDEBGnexus (producer-registration-task): registered with nexus successfully
17932024-12-14T01:16:25.807ZDEBGnexus (producer-registration-task): pausing until time to renew lease lease_duration = 600s wait_period = 150s
17942024-12-14T01:16:26.140ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 06366a99-c54c-4cfd-9538-1d21743ca1fd roles = RoleSet { roles: {} } uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
17952024-12-14T01:16:26.140ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 06366a99-c54c-4cfd-9538-1d21743ca1fd resource = Database result = Ok(()) uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
17962024-12-14T01:16:26.142ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2320 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 06366a99-c54c-4cfd-9538-1d21743ca1fd response_code = 200 uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
17972024-12-14T01:16:27.163ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 4cd70266-8bea-435f-b126-5c6efa008710 roles = RoleSet { roles: {} } uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
17982024-12-14T01:16:27.163ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 4cd70266-8bea-435f-b126-5c6efa008710 resource = Database result = Ok(()) uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
17992024-12-14T01:16:27.165ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2347 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 4cd70266-8bea-435f-b126-5c6efa008710 response_code = 200 uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
18002024-12-14T01:16:28.184ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 77212f24-fbba-4129-9476-6d726bf1f86f roles = RoleSet { roles: {} } uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
18012024-12-14T01:16:28.184ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 77212f24-fbba-4129-9476-6d726bf1f86f resource = Database result = Ok(()) uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
18022024-12-14T01:16:28.187ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2892 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 77212f24-fbba-4129-9476-6d726bf1f86f response_code = 200 uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
18032024-12-14T01:16:29.206ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 5c4a2f1a-dca1-45e1-abb8-1148e105e023 roles = RoleSet { roles: {} } uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
18042024-12-14T01:16:29.207ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 5c4a2f1a-dca1-45e1-abb8-1148e105e023 resource = Database result = Ok(()) uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
18052024-12-14T01:16:29.209ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2844 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 5c4a2f1a-dca1-45e1-abb8-1148e105e023 response_code = 200 uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
18062024-12-14T01:16:30.228ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 88fe65df-e00b-4a8e-b82e-04269bc34417 roles = RoleSet { roles: {} } uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
18072024-12-14T01:16:30.229ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 88fe65df-e00b-4a8e-b82e-04269bc34417 resource = Database result = Ok(()) uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
18082024-12-14T01:16:30.230ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2159 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 88fe65df-e00b-4a8e-b82e-04269bc34417 response_code = 200 uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
18092024-12-14T01:16:31.250ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = d9ac2960-31c1-4c5a-bd34-3ac174d118a9 roles = RoleSet { roles: {} } uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
18102024-12-14T01:16:31.251ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = d9ac2960-31c1-4c5a-bd34-3ac174d118a9 resource = Database result = Ok(()) uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
18112024-12-14T01:16:31.253ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2312 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = d9ac2960-31c1-4c5a-bd34-3ac174d118a9 response_code = 200 uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
18122024-12-14T01:16:32.273ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 23f7b9fd-be5b-4408-8b63-507d1e58dc26 roles = RoleSet { roles: {} } uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
18132024-12-14T01:16:32.274ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 23f7b9fd-be5b-4408-8b63-507d1e58dc26 resource = Database result = Ok(()) uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
18142024-12-14T01:16:32.276ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2487 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 23f7b9fd-be5b-4408-8b63-507d1e58dc26 response_code = 200 uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
18152024-12-14T01:16:33.296ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = b8e378da-f806-411b-97d0-142fa507c8ae roles = RoleSet { roles: {} } uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
18162024-12-14T01:16:33.297ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = b8e378da-f806-411b-97d0-142fa507c8ae resource = Database result = Ok(()) uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
18172024-12-14T01:16:33.299ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2649 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = b8e378da-f806-411b-97d0-142fa507c8ae response_code = 200 uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
18182024-12-14T01:16:33.462ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1527 local_addr = [fd00:1122:3344:101::c]:54941 method = GET remote_addr = [fd00:1122:3344:101::d]:43526 req_id = ad26c138-bb9e-4415-a3bb-b92eb02976ed response_code = 200 uri = /499bc009-7d23-4916-9be8-0a6a8f4e2f93
18192024-12-14T01:16:34.318ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = a02ed0e0-37f2-4f02-a39c-727514fa2c32 roles = RoleSet { roles: {} } uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
18202024-12-14T01:16:34.318ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = a02ed0e0-37f2-4f02-a39c-727514fa2c32 resource = Database result = Ok(()) uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
18212024-12-14T01:16:34.320ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2400 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = a02ed0e0-37f2-4f02-a39c-727514fa2c32 response_code = 200 uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
18222024-12-14T01:16:34.746ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 80 reason = Timeout
18232024-12-14T01:16:34.746ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 80 reason = Timeout
18242024-12-14T01:16:34.747ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
18252024-12-14T01:16:34.747ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
18262024-12-14T01:16:34.747ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
18272024-12-14T01:16:34.747ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
18282024-12-14T01:16:34.749ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18292024-12-14T01:16:34.750ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
18302024-12-14T01:16:34.750ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
18312024-12-14T01:16:34.751ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 4.825509ms iteration = 80 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
18322024-12-14T01:16:34.751ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
18332024-12-14T01:16:34.751ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
18342024-12-14T01:16:34.751ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
18352024-12-14T01:16:34.753ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
18362024-12-14T01:16:34.753ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
18372024-12-14T01:16:34.754ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18382024-12-14T01:16:34.757ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 8770bfcc-3a71-4738-a6b0-91008cadcb0f, lookup_type: ById(8770bfcc-3a71-4738-a6b0-91008cadcb0f) } result = Ok(())
18392024-12-14T01:16:34.757ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
18402024-12-14T01:16:34.757ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
18412024-12-14T01:16:34.768ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 8770bfcc-3a71-4738-a6b0-91008cadcb0f original_time_created = 2024-12-14 01:03:15.378067 UTC target_id = 8770bfcc-3a71-4738-a6b0-91008cadcb0f time_created = 2024-12-14 01:03:15.378067 UTC
18422024-12-14T01:16:34.768ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 21.166703ms iteration = 80 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"8770bfcc-3a71-4738-a6b0-91008cadcb0f","time_created":"2024-12-14T01:03:15.378067Z"}
18432024-12-14T01:16:35.340ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = bd59b511-a410-4c33-9d4e-da0a97926eb8 roles = RoleSet { roles: {} } uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
18442024-12-14T01:16:35.340ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = bd59b511-a410-4c33-9d4e-da0a97926eb8 resource = Database result = Ok(()) uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
18452024-12-14T01:16:35.342ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2255 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = bd59b511-a410-4c33-9d4e-da0a97926eb8 response_code = 200 uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
18462024-12-14T01:16:36.362ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = fe5d4ebe-008e-4872-9f41-f8ab524fa2c0 roles = RoleSet { roles: {} } uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
18472024-12-14T01:16:36.363ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = fe5d4ebe-008e-4872-9f41-f8ab524fa2c0 resource = Database result = Ok(()) uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
18482024-12-14T01:16:36.365ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2721 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = fe5d4ebe-008e-4872-9f41-f8ab524fa2c0 response_code = 200 uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
18492024-12-14T01:16:37.384ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 3cae5c4e-0808-4911-abff-3f72933aab9d roles = RoleSet { roles: {} } uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
18502024-12-14T01:16:37.385ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 3cae5c4e-0808-4911-abff-3f72933aab9d resource = Database result = Ok(()) uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
18512024-12-14T01:16:37.387ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2292 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 3cae5c4e-0808-4911-abff-3f72933aab9d response_code = 200 uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
18522024-12-14T01:16:38.406ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = fbac67b1-0f5f-4795-91a6-7779aaaa93cd roles = RoleSet { roles: {} } uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
18532024-12-14T01:16:38.406ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = fbac67b1-0f5f-4795-91a6-7779aaaa93cd resource = Database result = Ok(()) uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
18542024-12-14T01:16:38.408ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2234 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = fbac67b1-0f5f-4795-91a6-7779aaaa93cd response_code = 200 uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
18552024-12-14T01:16:39.426ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 1f9315bd-a0af-4b96-85c8-816faa6ed7cc roles = RoleSet { roles: {} } uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
18562024-12-14T01:16:39.426ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 1f9315bd-a0af-4b96-85c8-816faa6ed7cc resource = Database result = Ok(()) uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
18572024-12-14T01:16:39.428ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2206 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 1f9315bd-a0af-4b96-85c8-816faa6ed7cc response_code = 200 uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
18582024-12-14T01:16:40.447ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 3930ef94-ad9b-429d-8e8e-d3f57b967d90 roles = RoleSet { roles: {} } uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
18592024-12-14T01:16:40.448ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 3930ef94-ad9b-429d-8e8e-d3f57b967d90 resource = Database result = Ok(()) uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
18602024-12-14T01:16:40.450ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2253 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 3930ef94-ad9b-429d-8e8e-d3f57b967d90 response_code = 200 uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
18612024-12-14T01:16:41.468ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = d041e81b-5618-4a8c-9bc9-7e8b2e9433ce roles = RoleSet { roles: {} } uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
18622024-12-14T01:16:41.469ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = d041e81b-5618-4a8c-9bc9-7e8b2e9433ce resource = Database result = Ok(()) uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
18632024-12-14T01:16:41.470ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2136 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = d041e81b-5618-4a8c-9bc9-7e8b2e9433ce response_code = 200 uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
18642024-12-14T01:16:42.491ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = c89d004b-4fcd-4e34-9fb8-dcc88e064357 roles = RoleSet { roles: {} } uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
18652024-12-14T01:16:42.492ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = c89d004b-4fcd-4e34-9fb8-dcc88e064357 resource = Database result = Ok(()) uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
18662024-12-14T01:16:42.494ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2395 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = c89d004b-4fcd-4e34-9fb8-dcc88e064357 response_code = 200 uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
18672024-12-14T01:16:43.462ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1510 local_addr = [fd00:1122:3344:101::c]:54941 method = GET remote_addr = [fd00:1122:3344:101::d]:43526 req_id = 60356e23-c1ab-484f-adb5-a0c1d66797e5 response_code = 200 uri = /499bc009-7d23-4916-9be8-0a6a8f4e2f93
18682024-12-14T01:16:43.513ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = ce7dd0be-48f4-4487-979d-69b195fdc74b roles = RoleSet { roles: {} } uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
18692024-12-14T01:16:43.514ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = ce7dd0be-48f4-4487-979d-69b195fdc74b resource = Database result = Ok(()) uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
18702024-12-14T01:16:43.516ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2212 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = ce7dd0be-48f4-4487-979d-69b195fdc74b response_code = 200 uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
18712024-12-14T01:16:44.535ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = f3cb4aad-4671-411e-9974-ceb2af9dbb82 roles = RoleSet { roles: {} } uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
18722024-12-14T01:16:44.536ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = f3cb4aad-4671-411e-9974-ceb2af9dbb82 resource = Database result = Ok(()) uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
18732024-12-14T01:16:44.537ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2037 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = f3cb4aad-4671-411e-9974-ceb2af9dbb82 response_code = 200 uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
18742024-12-14T01:16:44.746ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 81 reason = Timeout
18752024-12-14T01:16:44.746ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
18762024-12-14T01:16:44.746ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 81 reason = Timeout
18772024-12-14T01:16:44.746ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
18782024-12-14T01:16:44.747ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
18792024-12-14T01:16:44.747ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
18802024-12-14T01:16:44.749ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
18812024-12-14T01:16:44.749ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
18822024-12-14T01:16:44.750ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18832024-12-14T01:16:44.752ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
18842024-12-14T01:16:44.752ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
18852024-12-14T01:16:44.753ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
18862024-12-14T01:16:44.755ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
18872024-12-14T01:16:44.755ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
18882024-12-14T01:16:44.757ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18892024-12-14T01:16:44.759ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 8770bfcc-3a71-4738-a6b0-91008cadcb0f, lookup_type: ById(8770bfcc-3a71-4738-a6b0-91008cadcb0f) } result = Ok(())
18902024-12-14T01:16:44.759ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
18912024-12-14T01:16:44.759ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
18922024-12-14T01:16:44.771ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 8770bfcc-3a71-4738-a6b0-91008cadcb0f original_time_created = 2024-12-14 01:03:15.378067 UTC target_id = 8770bfcc-3a71-4738-a6b0-91008cadcb0f time_created = 2024-12-14 01:03:15.378067 UTC
18932024-12-14T01:16:44.771ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 24.910451ms iteration = 81 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"8770bfcc-3a71-4738-a6b0-91008cadcb0f","time_created":"2024-12-14T01:03:15.378067Z"}
18942024-12-14T01:16:44.777ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 30.210004ms iteration = 81 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
18952024-12-14T01:16:45.557ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = a8a8448f-4317-4532-8879-3af92a34fca1 roles = RoleSet { roles: {} } uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
18962024-12-14T01:16:45.557ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = a8a8448f-4317-4532-8879-3af92a34fca1 resource = Database result = Ok(()) uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
18972024-12-14T01:16:45.559ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2312 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = a8a8448f-4317-4532-8879-3af92a34fca1 response_code = 200 uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
18982024-12-14T01:16:46.580ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = bc632afa-1782-4200-a294-0c16a89c0ea8 roles = RoleSet { roles: {} } uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
18992024-12-14T01:16:46.580ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = bc632afa-1782-4200-a294-0c16a89c0ea8 resource = Database result = Ok(()) uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
19002024-12-14T01:16:46.582ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2714 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = bc632afa-1782-4200-a294-0c16a89c0ea8 response_code = 200 uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
19012024-12-14T01:16:47.601ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 7985ec91-0a11-4dfd-97f5-fd56ce75579b roles = RoleSet { roles: {} } uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
19022024-12-14T01:16:47.601ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 7985ec91-0a11-4dfd-97f5-fd56ce75579b resource = Database result = Ok(()) uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
19032024-12-14T01:16:47.603ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2377 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 7985ec91-0a11-4dfd-97f5-fd56ce75579b response_code = 200 uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
19042024-12-14T01:16:48.622ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 2aa98fdb-b3d5-44c4-9fed-79d9d6a1185c roles = RoleSet { roles: {} } uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
19052024-12-14T01:16:48.623ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 2aa98fdb-b3d5-44c4-9fed-79d9d6a1185c resource = Database result = Ok(()) uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
19062024-12-14T01:16:48.625ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2073 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 2aa98fdb-b3d5-44c4-9fed-79d9d6a1185c response_code = 200 uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
19072024-12-14T01:16:49.644ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 63568c7a-3c8b-4e08-91a6-e31c8bedd90e roles = RoleSet { roles: {} } uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
19082024-12-14T01:16:49.644ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 63568c7a-3c8b-4e08-91a6-e31c8bedd90e resource = Database result = Ok(()) uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
19092024-12-14T01:16:49.646ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2240 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 63568c7a-3c8b-4e08-91a6-e31c8bedd90e response_code = 200 uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
19102024-12-14T01:16:50.179ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:43741 req_id = 1fedd543-a764-4389-9168-8d71ea12546a roles = RoleSet { roles: {} } uri = /nat/ipv4/changeset/10?limit=100
19112024-12-14T01:16:50.180ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:43741 req_id = 1fedd543-a764-4389-9168-8d71ea12546a resource = Database result = Ok(()) uri = /nat/ipv4/changeset/10?limit=100
19122024-12-14T01:16:50.182ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 3533 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:43741 req_id = 1fedd543-a764-4389-9168-8d71ea12546a response_code = 200 uri = /nat/ipv4/changeset/10?limit=100
19132024-12-14T01:16:50.665ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 3f157b05-e456-454d-86e8-19f304a79e1c roles = RoleSet { roles: {} } uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
19142024-12-14T01:16:50.665ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 3f157b05-e456-454d-86e8-19f304a79e1c resource = Database result = Ok(()) uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
19152024-12-14T01:16:50.668ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2748 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 3f157b05-e456-454d-86e8-19f304a79e1c response_code = 200 uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
19162024-12-14T01:16:51.688ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 92937c4c-2dfc-4322-8afc-e75cfaba0ee6 roles = RoleSet { roles: {} } uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
19172024-12-14T01:16:51.689ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 92937c4c-2dfc-4322-8afc-e75cfaba0ee6 resource = Database result = Ok(()) uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
19182024-12-14T01:16:51.691ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2341 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 92937c4c-2dfc-4322-8afc-e75cfaba0ee6 response_code = 200 uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
19192024-12-14T01:16:52.712ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = f4367d92-836b-4331-872a-545d90f2869c roles = RoleSet { roles: {} } uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
19202024-12-14T01:16:52.713ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = f4367d92-836b-4331-872a-545d90f2869c resource = Database result = Ok(()) uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
19212024-12-14T01:16:52.714ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2297 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = f4367d92-836b-4331-872a-545d90f2869c response_code = 200 uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
19222024-12-14T01:16:53.462ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1533 local_addr = [fd00:1122:3344:101::c]:54941 method = GET remote_addr = [fd00:1122:3344:101::d]:43526 req_id = 62707240-8ea7-4281-bb35-13d40b1a517a response_code = 200 uri = /499bc009-7d23-4916-9be8-0a6a8f4e2f93
19232024-12-14T01:16:53.733ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = ca93c385-8852-4c6a-a660-aa22e1957a88 roles = RoleSet { roles: {} } uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
19242024-12-14T01:16:53.733ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = ca93c385-8852-4c6a-a660-aa22e1957a88 resource = Database result = Ok(()) uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
19252024-12-14T01:16:53.735ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2537 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = ca93c385-8852-4c6a-a660-aa22e1957a88 response_code = 200 uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
19262024-12-14T01:16:54.746ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 28 reason = Timeout
19272024-12-14T01:16:54.747ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 28 reason = Timeout
19282024-12-14T01:16:54.747ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
19292024-12-14T01:16:54.747ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
19302024-12-14T01:16:54.747ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 28 reason = Timeout
19312024-12-14T01:16:54.747ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
19322024-12-14T01:16:54.747ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 32 reason = Timeout
19332024-12-14T01:16:54.747ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 28 reason = Timeout
19342024-12-14T01:16:54.747ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
19352024-12-14T01:16:54.747ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 28 reason = Timeout
19362024-12-14T01:16:54.747ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 82 reason = Timeout
19372024-12-14T01:16:54.747ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 28 reason = Timeout
19382024-12-14T01:16:54.747ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
19392024-12-14T01:16:54.747ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
19402024-12-14T01:16:54.747ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
19412024-12-14T01:16:54.747ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 28 reason = Timeout
19422024-12-14T01:16:54.747ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
19432024-12-14T01:16:54.747ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 28 reason = Timeout
19442024-12-14T01:16:54.747ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
19452024-12-14T01:16:54.747ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
19462024-12-14T01:16:54.747ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 28 reason = Timeout
19472024-12-14T01:16:54.747ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
19482024-12-14T01:16:54.747ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 30 reason = Timeout
19492024-12-14T01:16:54.747ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
19502024-12-14T01:16:54.747ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
19512024-12-14T01:16:54.747ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 28 reason = Timeout
19522024-12-14T01:16:54.747ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
19532024-12-14T01:16:54.747ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 82 reason = Timeout
19542024-12-14T01:16:54.747ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
19552024-12-14T01:16:54.747ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 28 reason = Timeout
19562024-12-14T01:16:54.747ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
19572024-12-14T01:16:54.747ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 28 reason = Timeout
19582024-12-14T01:16:54.747ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
19592024-12-14T01:16:54.747ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 29 reason = Timeout
19602024-12-14T01:16:54.747ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
19612024-12-14T01:16:54.747ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
19622024-12-14T01:16:54.747ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
19632024-12-14T01:16:54.747ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
19642024-12-14T01:16:54.747ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
19652024-12-14T01:16:54.747ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
19662024-12-14T01:16:54.747ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
19672024-12-14T01:16:54.747ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
19682024-12-14T01:16:54.747ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
19692024-12-14T01:16:54.747ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
19702024-12-14T01:16:54.748ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
19712024-12-14T01:16:54.748ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
19722024-12-14T01:16:54.748ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
19732024-12-14T01:16:54.748ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
19742024-12-14T01:16:54.748ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
19752024-12-14T01:16:54.748ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
19762024-12-14T01:16:54.748ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
19772024-12-14T01:16:54.749ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19782024-12-14T01:16:54.750ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-7e423e95-464a-471f-a0d9-4799977f0b0f.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-7e423e95-464a-471f-a0d9-4799977f0b0f.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1577, tv_nsec: 579118918 } })
19792024-12-14T01:16:54.750ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1103
19802024-12-14T01:16:54.750ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.607994ms iteration = 28 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
19812024-12-14T01:16:54.751ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19822024-12-14T01:16:54.752ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
19832024-12-14T01:16:54.752ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
19842024-12-14T01:16:54.752ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
19852024-12-14T01:16:54.753ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19862024-12-14T01:16:54.753ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
19872024-12-14T01:16:54.753ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
19882024-12-14T01:16:54.754ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
19892024-12-14T01:16:54.755ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = d9397eda-7feb-41c3-b470-8daf6c61e816 roles = RoleSet { roles: {} } uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
19902024-12-14T01:16:54.755ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = d9397eda-7feb-41c3-b470-8daf6c61e816 resource = Database result = Ok(()) uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
19912024-12-14T01:16:54.756ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19922024-12-14T01:16:54.756ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
19932024-12-14T01:16:54.756ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
19942024-12-14T01:16:54.756ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
19952024-12-14T01:16:54.758ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
19962024-12-14T01:16:54.758ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
19972024-12-14T01:16:54.758ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
19982024-12-14T01:16:54.759ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 12.159862ms iteration = 28 status = {"errors":[],"records_set_to_done":[]}
19992024-12-14T01:16:54.759ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20002024-12-14T01:16:54.759ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-7e423e95-464a-471f-a0d9-4799977f0b0f.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-7e423e95-464a-471f-a0d9-4799977f0b0f.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1577, tv_nsec: 588943073 } })
20012024-12-14T01:16:54.760ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1103
20022024-12-14T01:16:54.761ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 6839 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = d9397eda-7feb-41c3-b470-8daf6c61e816 response_code = 200 uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
20032024-12-14T01:16:54.762ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-7e423e95-464a-471f-a0d9-4799977f0b0f.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-7e423e95-464a-471f-a0d9-4799977f0b0f.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1577, tv_nsec: 591036843 } })
20042024-12-14T01:16:54.762ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1103 rack_id = f30fd360-30ae-4088-9821-aa98c2bc8c09
20052024-12-14T01:16:54.764ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
20062024-12-14T01:16:54.764ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
20072024-12-14T01:16:54.765ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
20082024-12-14T01:16:54.766ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
20092024-12-14T01:16:54.766ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
20102024-12-14T01:16:54.768ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
20112024-12-14T01:16:54.768ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
20122024-12-14T01:16:54.769ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
20132024-12-14T01:16:54.769ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
20142024-12-14T01:16:54.770ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20152024-12-14T01:16:54.771ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20162024-12-14T01:16:54.772ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
20172024-12-14T01:16:54.772ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
20182024-12-14T01:16:54.773ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
20192024-12-14T01:16:54.776ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 8770bfcc-3a71-4738-a6b0-91008cadcb0f, lookup_type: ById(8770bfcc-3a71-4738-a6b0-91008cadcb0f) } result = Ok(())
20202024-12-14T01:16:54.776ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
20212024-12-14T01:16:54.776ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
20222024-12-14T01:16:54.786ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
20232024-12-14T01:16:54.786ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
20242024-12-14T01:16:54.787ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
20252024-12-14T01:16:54.787ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
20262024-12-14T01:16:54.789ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 42.616033ms iteration = 28 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
20272024-12-14T01:16:54.789ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
20282024-12-14T01:16:54.790ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
20292024-12-14T01:16:54.791ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 8770bfcc-3a71-4738-a6b0-91008cadcb0f original_time_created = 2024-12-14 01:03:15.378067 UTC target_id = 8770bfcc-3a71-4738-a6b0-91008cadcb0f time_created = 2024-12-14 01:03:15.378067 UTC
20302024-12-14T01:16:54.791ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 44.326723ms iteration = 82 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"8770bfcc-3a71-4738-a6b0-91008cadcb0f","time_created":"2024-12-14T01:03:15.378067Z"}
20312024-12-14T01:16:54.792ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20322024-12-14T01:16:54.793ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 47.043271ms iteration = 28 status = {"errors":[],"garbage_collect_requested":[]}
20332024-12-14T01:16:54.794ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
20342024-12-14T01:16:54.794ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
20352024-12-14T01:16:54.794ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
20362024-12-14T01:16:54.795ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
20372024-12-14T01:16:54.795ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
20382024-12-14T01:16:54.798ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
20392024-12-14T01:16:54.798ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
20402024-12-14T01:16:54.798ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
20412024-12-14T01:16:54.798ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
20422024-12-14T01:16:54.798ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
20432024-12-14T01:16:54.798ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
20442024-12-14T01:16:54.800ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
20452024-12-14T01:16:54.800ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 53.070356ms iteration = 28 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
20462024-12-14T01:16:54.800ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
20472024-12-14T01:16:54.800ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
20482024-12-14T01:16:54.800ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
20492024-12-14T01:16:54.800ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
20502024-12-14T01:16:54.801ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
20512024-12-14T01:16:54.801ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
20522024-12-14T01:16:54.805ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20532024-12-14T01:16:54.806ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
20542024-12-14T01:16:54.806ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
20552024-12-14T01:16:54.807ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
20562024-12-14T01:16:54.807ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
20572024-12-14T01:16:54.808ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
20582024-12-14T01:16:54.822ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 75.33178ms iteration = 82 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
20592024-12-14T01:16:54.827ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
20602024-12-14T01:16:54.828ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
20612024-12-14T01:16:54.830ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 82.94896ms iteration = 28 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[]}
20622024-12-14T01:16:54.830ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 83.60962ms iteration = 28 status = {"errors":[],"requests_created_ok":[],"start_invoked_ok":[]}
20632024-12-14T01:16:54.841ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
20642024-12-14T01:16:54.841ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
20652024-12-14T01:16:54.841ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 94.67526ms iteration = 28 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
20662024-12-14T01:16:54.843ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
20672024-12-14T01:16:54.843ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
20682024-12-14T01:16:54.848ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20692024-12-14T01:16:54.851ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 7e423e95-464a-471f-a0d9-4799977f0b0f, lookup_type: ById(7e423e95-464a-471f-a0d9-4799977f0b0f) } result = Ok(())
20702024-12-14T01:16:54.851ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
20712024-12-14T01:16:54.851ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
20722024-12-14T01:16:54.860ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1111 rack_id = f30fd360-30ae-4088-9821-aa98c2bc8c09 zone_address = fd00:1122:3344:101::2
20732024-12-14T01:16:54.860ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = f30fd360-30ae-4088-9821-aa98c2bc8c09 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
20742024-12-14T01:16:54.861ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = f30fd360-30ae-4088-9821-aa98c2bc8c09 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "f5478211-cf77-4844-9dc0-190af8307854", "content-length": "26", "date": "Sat, 14 Dec 2024 01:16:54 GMT"} })
20752024-12-14T01:16:54.861ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1116 rack_id = f30fd360-30ae-4088-9821-aa98c2bc8c09 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
20762024-12-14T01:16:54.862ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = f30fd360-30ae-4088-9821-aa98c2bc8c09
20772024-12-14T01:16:54.881ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 134.698988ms iteration = 32 status = {"physical_disks_added":0}
20782024-12-14T01:16:54.912ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
20792024-12-14T01:16:54.912ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
20802024-12-14T01:16:54.912ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "54fc22e6-a7e3-4080-985a-6bc031860a00", "content-length": "26", "date": "Sat, 14 Dec 2024 01:16:54 GMT"} })
20812024-12-14T01:16:54.913ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
20822024-12-14T01:16:54.914ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
20832024-12-14T01:16:54.942ZINFOnexus (BackgroundTasks): VPC route manager sled 7e423e95-464a-471f-a0d9-4799977f0b0f background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
20842024-12-14T01:16:54.943ZDEBGnexus (BackgroundTasks): client request SledAgent = 7e423e95-464a-471f-a0d9-4799977f0b0f background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
20852024-12-14T01:16:54.943ZDEBGnexus (BackgroundTasks): client response SledAgent = 7e423e95-464a-471f-a0d9-4799977f0b0f background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "810e6c97-6198-4319-9264-fb27bdd2eb7c", "content-length": "372", "date": "Sat, 14 Dec 2024 01:16:54 GMT"} })
20862024-12-14T01:16:54.943ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
20872024-12-14T01:16:54.943ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
20882024-12-14T01:16:54.944ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20892024-12-14T01:16:54.945ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 198.371991ms iteration = 28 status = 0
20902024-12-14T01:16:54.946ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
20912024-12-14T01:16:54.946ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
20922024-12-14T01:16:54.946ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
20932024-12-14T01:16:54.947ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
20942024-12-14T01:16:54.947ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
20952024-12-14T01:16:54.947ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "9940b7c5-e399-426d-b556-1584dbbbc543", "content-length": "26", "date": "Sat, 14 Dec 2024 01:16:54 GMT"} })
20962024-12-14T01:16:54.947ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
20972024-12-14T01:16:54.948ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20982024-12-14T01:16:54.949ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
20992024-12-14T01:16:54.949ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
21002024-12-14T01:16:54.949ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
21012024-12-14T01:16:54.950ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
21022024-12-14T01:16:54.987ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = buskin
21032024-12-14T01:16:54.987ZDEBGnexus (BackgroundTasks): client request SledAgent = 7e423e95-464a-471f-a0d9-4799977f0b0f background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
21042024-12-14T01:16:54.988ZDEBGnexus (BackgroundTasks): client response SledAgent = 7e423e95-464a-471f-a0d9-4799977f0b0f background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "971ab2aa-6563-43c6-b61e-13f80e4c0835", "date": "Sat, 14 Dec 2024 01:16:54 GMT"} })
21052024-12-14T01:16:54.988ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
21062024-12-14T01:16:54.988ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
21072024-12-14T01:16:54.990ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
21082024-12-14T01:16:54.990ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
21092024-12-14T01:16:54.993ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
21102024-12-14T01:16:54.994ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
21112024-12-14T01:16:54.997ZINFOnexus (BackgroundTasks): VPC route manager sled 7e423e95-464a-471f-a0d9-4799977f0b0f push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
21122024-12-14T01:16:54.997ZDEBGnexus (BackgroundTasks): client request SledAgent = 7e423e95-464a-471f-a0d9-4799977f0b0f background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
21132024-12-14T01:16:54.997ZDEBGnexus (BackgroundTasks): client response SledAgent = 7e423e95-464a-471f-a0d9-4799977f0b0f background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "317de31e-50c1-4588-b6ea-8c443fed22ca", "date": "Sat, 14 Dec 2024 01:16:54 GMT"} })
21142024-12-14T01:16:54.999ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 252.380833ms iteration = 30 status = {}
21152024-12-14T01:16:55.009ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
21162024-12-14T01:16:55.009ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "f62bbb02-f1b1-44b9-b8a4-fa5c3aa31035", "content-length": "2", "date": "Sat, 14 Dec 2024 01:16:54 GMT"} })
21172024-12-14T01:16:55.011ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
21182024-12-14T01:16:55.011ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
21192024-12-14T01:16:55.013ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 266.655917ms iteration = 28 status = null
21202024-12-14T01:16:55.025ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 278.705636ms iteration = 28 status = {}
21212024-12-14T01:16:55.077ZDEBGnexus (BackgroundTasks): client request SledAgent = 7e423e95-464a-471f-a0d9-4799977f0b0f background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
21222024-12-14T01:16:55.077ZDEBGnexus (BackgroundTasks): client response SledAgent = 7e423e95-464a-471f-a0d9-4799977f0b0f background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "c218ec99-b8de-4265-b6a3-702f0042df21", "content-length": "691", "date": "Sat, 14 Dec 2024 01:16:54 GMT"} })
21232024-12-14T01:16:55.077ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = buskin
21242024-12-14T01:16:55.077ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = buskin
21252024-12-14T01:16:55.077ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = buskin
21262024-12-14T01:16:55.078ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 331.274877ms iteration = 29 status = {}
21272024-12-14T01:16:55.091ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
21282024-12-14T01:16:55.091ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
21292024-12-14T01:16:55.094ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: bb1a4b50-730e-48ee-a311-8a9b9d7bcb73, rack_id: f30fd360-30ae-4088-9821-aa98c2bc8c09, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(bb9f5a1a-5708-4d58-ba4f-c62102236426) } rack_id = f30fd360-30ae-4088-9821-aa98c2bc8c09 switch_location = Switch0
21302024-12-14T01:16:55.094ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
21312024-12-14T01:16:55.094ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
21322024-12-14T01:16:55.103ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = f30fd360-30ae-4088-9821-aa98c2bc8c09 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
21332024-12-14T01:16:55.103ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = f30fd360-30ae-4088-9821-aa98c2bc8c09 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "52f210c3-2f3d-4ffd-9e25-556c8a2a1505", "content-length": "134", "date": "Sat, 14 Dec 2024 01:16:54 GMT"} })
21342024-12-14T01:16:55.103ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1746 rack_id = f30fd360-30ae-4088-9821-aa98c2bc8c09 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
21352024-12-14T01:16:55.103ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1773 rack_id = f30fd360-30ae-4088-9821-aa98c2bc8c09 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = bb9f5a1a-5708-4d58-ba4f-c62102236426
21362024-12-14T01:16:55.103ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = f30fd360-30ae-4088-9821-aa98c2bc8c09 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
21372024-12-14T01:16:55.103ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = f30fd360-30ae-4088-9821-aa98c2bc8c09 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
21382024-12-14T01:16:55.103ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = f30fd360-30ae-4088-9821-aa98c2bc8c09 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
21392024-12-14T01:16:55.103ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = f30fd360-30ae-4088-9821-aa98c2bc8c09 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
21402024-12-14T01:16:55.103ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = f30fd360-30ae-4088-9821-aa98c2bc8c09 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
21412024-12-14T01:16:55.103ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1920 rack_id = f30fd360-30ae-4088-9821-aa98c2bc8c09 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
21422024-12-14T01:16:55.103ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = f30fd360-30ae-4088-9821-aa98c2bc8c09 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
21432024-12-14T01:16:55.103ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = f30fd360-30ae-4088-9821-aa98c2bc8c09 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
21442024-12-14T01:16:55.103ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = f30fd360-30ae-4088-9821-aa98c2bc8c09
21452024-12-14T01:16:55.103ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
21462024-12-14T01:16:55.104ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
21472024-12-14T01:16:55.106ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = f30fd360-30ae-4088-9821-aa98c2bc8c09 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
21482024-12-14T01:16:55.106ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = f30fd360-30ae-4088-9821-aa98c2bc8c09 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "fa1062b6-1b6a-4d21-9bc8-0fd2679c9083", "content-length": "2", "date": "Sat, 14 Dec 2024 01:16:55 GMT"} })
21492024-12-14T01:16:55.106ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = f30fd360-30ae-4088-9821-aa98c2bc8c09 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
21502024-12-14T01:16:55.106ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = f30fd360-30ae-4088-9821-aa98c2bc8c09 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "63b3923f-9495-4ce8-a35c-5e11b88f05f0", "content-length": "52", "date": "Sat, 14 Dec 2024 01:16:55 GMT"} })
21512024-12-14T01:16:55.106ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.1.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = f30fd360-30ae-4088-9821-aa98c2bc8c09 switch_location = Switch0
21522024-12-14T01:16:55.106ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = f30fd360-30ae-4088-9821-aa98c2bc8c09 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
21532024-12-14T01:16:55.198ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = f30fd360-30ae-4088-9821-aa98c2bc8c09 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "812cf97b-c92d-4bed-8a59-bbb779a5e792", "date": "Sat, 14 Dec 2024 01:16:54 GMT"} })
21542024-12-14T01:16:55.198ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = f30fd360-30ae-4088-9821-aa98c2bc8c09 switch = Switch0
21552024-12-14T01:16:55.198ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = f30fd360-30ae-4088-9821-aa98c2bc8c09 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
21562024-12-14T01:16:55.198ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = f30fd360-30ae-4088-9821-aa98c2bc8c09 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "4df01737-2729-4f54-ae76-19af2b5cfba1", "content-length": "682", "date": "Sat, 14 Dec 2024 01:16:54 GMT"} })
21572024-12-14T01:16:55.198ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
21582024-12-14T01:16:55.199ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
21592024-12-14T01:16:55.203ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
21602024-12-14T01:16:55.203ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
21612024-12-14T01:16:55.211ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
21622024-12-14T01:16:55.211ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
21632024-12-14T01:16:55.213ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
21642024-12-14T01:16:55.214ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
21652024-12-14T01:16:55.221ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 474.887973ms iteration = 28 status = {}
21662024-12-14T01:16:55.784ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 7d4f41d7-e4ea-46a7-b9de-07d2b1a7111c roles = RoleSet { roles: {} } uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
21672024-12-14T01:16:55.785ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 7d4f41d7-e4ea-46a7-b9de-07d2b1a7111c resource = Database result = Ok(()) uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f
21682024-12-14T01:16:55.787ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2617 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:50234 req_id = 7d4f41d7-e4ea-46a7-b9de-07d2b1a7111c response_code = 200 uri = /probes/7e423e95-464a-471f-a0d9-4799977f0b0f