Max level shown:
12024-02-04T18:30:04.018ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 1337 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:47021 req_id = ee0f84a3-d5b3-445a-b00e-87bd6663c889 response_code = 200 uri = /metrics/collect/092d8721-ad83-4ea8-800d-5dfcaa02a610
22024-02-04T18:30:05.763ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 39 reason = Timeout
32024-02-04T18:30:05.763ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
42024-02-04T18:30:05.763ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
52024-02-04T18:30:05.765ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
62024-02-04T18:30:05.768ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
72024-02-04T18:30:05.768ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
82024-02-04T18:30:05.768ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
92024-02-04T18:30:05.769ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 6.531983ms iteration = 39
102024-02-04T18:30:14.018ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 1618 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:47021 req_id = 0406e308-4326-4734-8830-9d1126c7dc25 response_code = 200 uri = /metrics/collect/092d8721-ad83-4ea8-800d-5dfcaa02a610
112024-02-04T18:30:15.763ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 40 reason = Timeout
122024-02-04T18:30:15.763ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
132024-02-04T18:30:15.763ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
142024-02-04T18:30:15.765ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
152024-02-04T18:30:15.767ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
162024-02-04T18:30:15.767ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
172024-02-04T18:30:15.767ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
182024-02-04T18:30:15.769ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 6.119098ms iteration = 40
192024-02-04T18:30:24.019ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 2109 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:47021 req_id = d0fc1911-95cd-4350-9a89-a08f293d8d5a response_code = 200 uri = /metrics/collect/092d8721-ad83-4ea8-800d-5dfcaa02a610
202024-02-04T18:30:25.752ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 15 reason = Timeout
212024-02-04T18:30:25.752ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
222024-02-04T18:30:25.753ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
232024-02-04T18:30:25.754ZDEBGnexus (DpdClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
242024-02-04T18:30:25.754ZDEBGnexus (DpdClient): client response result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "2be9a97b-ccd4-4022-93fb-599d7f80ef03", "content-length": "2", "date": "Sun, 04 Feb 2024 18:30:25 GMT"} })
252024-02-04T18:30:25.754ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
262024-02-04T18:30:25.754ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
272024-02-04T18:30:25.756ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 3.337042ms iteration = 15
282024-02-04T18:30:25.764ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 41 reason = Timeout
292024-02-04T18:30:25.764ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 15 reason = Timeout
302024-02-04T18:30:25.764ZWARNnexus (BackgroundTasks): phantom disk task started background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:46
312024-02-04T18:30:25.764ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 15 reason = Timeout
322024-02-04T18:30:25.764ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
332024-02-04T18:30:25.764ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
342024-02-04T18:30:25.764ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 15 reason = Timeout
352024-02-04T18:30:25.764ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
362024-02-04T18:30:25.764ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
372024-02-04T18:30:25.764ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
382024-02-04T18:30:25.764ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
392024-02-04T18:30:25.765ZDEBGnexus: client request MgAdminClient = [fd00:1122:3344:101::2]:4676 body = None method = GET uri = http://[fd00:1122:3344:101::2]:4676/bfd/peers
402024-02-04T18:30:25.765ZDEBGnexus: client response MgAdminClient = [fd00:1122:3344:101::2]:4676 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(4676), path: "/bfd/peers", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "919d2f43-9824-4ffb-99e8-0ca283bf6841", "content-length": "2", "date": "Sun, 04 Feb 2024 18:30:25 GMT"} })
412024-02-04T18:30:25.765ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 1.739012ms iteration = 15
422024-02-04T18:30:25.766ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
432024-02-04T18:30:25.766ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
442024-02-04T18:30:25.766ZWARNnexus (BackgroundTasks): phantom disk task done background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:92
452024-02-04T18:30:25.766ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 2.779285ms iteration = 15
462024-02-04T18:30:25.769ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
472024-02-04T18:30:25.769ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
482024-02-04T18:30:25.769ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
492024-02-04T18:30:25.769ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
502024-02-04T18:30:25.769ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
512024-02-04T18:30:25.769ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
522024-02-04T18:30:25.771ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 7.028418ms iteration = 41
532024-02-04T18:30:25.771ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
542024-02-04T18:30:25.771ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
552024-02-04T18:30:25.784ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
562024-02-04T18:30:25.785ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
572024-02-04T18:30:25.786ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
582024-02-04T18:30:25.786ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
592024-02-04T18:30:25.787ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
602024-02-04T18:30:25.789ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 339b514e-8533-421a-a5ed-83b7bd206824, lookup_type: ById(339b514e-8533-421a-a5ed-83b7bd206824) } result = Ok(())
612024-02-04T18:30:25.789ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
622024-02-04T18:30:25.789ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
632024-02-04T18:30:25.790ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 26.446244ms iteration = 15
642024-02-04T18:30:26.177ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:44921 req_id = d7671309-b5e3-42cd-b09f-5e1247801233 roles = RoleSet { roles: {} } uri = /nat/ipv4/changeset/12?limit=100
652024-02-04T18:30:26.178ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:44921 req_id = d7671309-b5e3-42cd-b09f-5e1247801233 resource = Database result = Ok(()) uri = /nat/ipv4/changeset/12?limit=100
662024-02-04T18:30:26.179ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 1961 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:44921 req_id = d7671309-b5e3-42cd-b09f-5e1247801233 response_code = 200 uri = /nat/ipv4/changeset/12?limit=100
672024-02-04T18:30:34.018ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 1329 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:47021 req_id = 4eed3f3f-818e-48d2-840b-cab244e7dce6 response_code = 200 uri = /metrics/collect/092d8721-ad83-4ea8-800d-5dfcaa02a610
682024-02-04T18:30:35.763ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 42 reason = Timeout
692024-02-04T18:30:35.763ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
702024-02-04T18:30:35.763ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
712024-02-04T18:30:35.765ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
722024-02-04T18:30:35.767ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
732024-02-04T18:30:35.767ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
742024-02-04T18:30:35.767ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
752024-02-04T18:30:35.769ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 5.663931ms iteration = 42
762024-02-04T18:30:36.190ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:44921 req_id = f3276a3f-e2e9-4c0a-9e58-abe76a275644 roles = RoleSet { roles: {} } uri = /nat/ipv4/changeset/12?limit=100
772024-02-04T18:30:36.191ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:44921 req_id = f3276a3f-e2e9-4c0a-9e58-abe76a275644 resource = Database result = Ok(()) uri = /nat/ipv4/changeset/12?limit=100
782024-02-04T18:30:36.193ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 3486 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:44921 req_id = f3276a3f-e2e9-4c0a-9e58-abe76a275644 response_code = 200 uri = /nat/ipv4/changeset/12?limit=100
792024-02-04T18:30:36.193ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:44921 req_id = 7c25e6bc-dda3-475a-8edf-2718b17ae7e5 roles = RoleSet { roles: {} } uri = /nat/ipv4/changeset/14?limit=100
802024-02-04T18:30:36.194ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:44921 req_id = 7c25e6bc-dda3-475a-8edf-2718b17ae7e5 resource = Database result = Ok(()) uri = /nat/ipv4/changeset/14?limit=100
812024-02-04T18:30:36.195ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 1738 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:44921 req_id = 7c25e6bc-dda3-475a-8edf-2718b17ae7e5 response_code = 200 uri = /nat/ipv4/changeset/14?limit=100
822024-02-04T18:30:43.152ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:765 local_addr = [fd00:1122:3344:101::b]:12221 remote_addr = [fd00:1122:3344:101::1]:51916
832024-02-04T18:30:43.161ZINFOnexus: received new runtime state from sled agent file = nexus/src/app/instance.rs:1472 instance_id = 978da889-c1eb-4016-b182-d8c4346a3c1d instance_state = InstanceRuntimeState { propolis_id: Some(a99c950c-4d50-4241-8074-58986809f024), dst_propolis_id: None, migration_id: None, gen: Generation(3), time_updated: 2024-02-04T18:30:35.410980497Z } propolis_id = a99c950c-4d50-4241-8074-58986809f024 vmm_state = VmmRuntimeState { state: Starting, gen: Generation(2), time_updated: 2024-02-04T18:30:43.150811735Z }
842024-02-04T18:30:43.167ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:51916 req_id = 84670375-f062-425e-9907-f41898766537 roles = RoleSet { roles: {} } uri = /instances/978da889-c1eb-4016-b182-d8c4346a3c1d
852024-02-04T18:30:43.167ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:51916 req_id = 84670375-f062-425e-9907-f41898766537 resource = Database result = Ok(()) uri = /instances/978da889-c1eb-4016-b182-d8c4346a3c1d
862024-02-04T18:30:43.177ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:51916 req_id = 84670375-f062-425e-9907-f41898766537 roles = RoleSet { roles: {} } uri = /instances/978da889-c1eb-4016-b182-d8c4346a3c1d
872024-02-04T18:30:43.177ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:51916 req_id = 84670375-f062-425e-9907-f41898766537 resource = Database result = Ok(()) uri = /instances/978da889-c1eb-4016-b182-d8c4346a3c1d
882024-02-04T18:30:43.178ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:51916 req_id = 84670375-f062-425e-9907-f41898766537 roles = RoleSet { roles: {} } uri = /instances/978da889-c1eb-4016-b182-d8c4346a3c1d
892024-02-04T18:30:43.179ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:51916 req_id = 84670375-f062-425e-9907-f41898766537 resource = Database result = Ok(()) uri = /instances/978da889-c1eb-4016-b182-d8c4346a3c1d
902024-02-04T18:30:43.181ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:51916 req_id = 84670375-f062-425e-9907-f41898766537 roles = RoleSet { roles: {} } uri = /instances/978da889-c1eb-4016-b182-d8c4346a3c1d
912024-02-04T18:30:43.181ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:51916 req_id = 84670375-f062-425e-9907-f41898766537 resource = Database result = Ok(()) uri = /instances/978da889-c1eb-4016-b182-d8c4346a3c1d
922024-02-04T18:30:43.182ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:51916 req_id = 84670375-f062-425e-9907-f41898766537 roles = RoleSet { roles: {} } uri = /instances/978da889-c1eb-4016-b182-d8c4346a3c1d
932024-02-04T18:30:43.182ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:51916 req_id = 84670375-f062-425e-9907-f41898766537 resource = Database result = Ok(()) uri = /instances/978da889-c1eb-4016-b182-d8c4346a3c1d
942024-02-04T18:30:43.184ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:51916 req_id = 84670375-f062-425e-9907-f41898766537 roles = RoleSet { roles: {} } uri = /instances/978da889-c1eb-4016-b182-d8c4346a3c1d
952024-02-04T18:30:43.184ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:51916 req_id = 84670375-f062-425e-9907-f41898766537 resource = Database result = Ok(()) uri = /instances/978da889-c1eb-4016-b182-d8c4346a3c1d
962024-02-04T18:30:43.185ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:51916 req_id = 84670375-f062-425e-9907-f41898766537 roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } uri = /instances/978da889-c1eb-4016-b182-d8c4346a3c1d
972024-02-04T18:30:43.189ZDEBGnexus (dropshot_internal): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:51916 req_id = 84670375-f062-425e-9907-f41898766537 resource = Instance { parent: Project { parent: Silo { parent: Fleet, key: 7eaca1c5-9568-4aa5-a1b2-cb905947fbfb, lookup_type: ById(7eaca1c5-9568-4aa5-a1b2-cb905947fbfb) }, key: 638a1648-cc73-4192-b4a6-cacb5faecf9d, lookup_type: ById(638a1648-cc73-4192-b4a6-cacb5faecf9d) }, key: 978da889-c1eb-4016-b182-d8c4346a3c1d, lookup_type: ById(978da889-c1eb-4016-b182-d8c4346a3c1d) } result = Ok(()) uri = /instances/978da889-c1eb-4016-b182-d8c4346a3c1d
982024-02-04T18:30:43.189ZDEBGnexus: instance state generation already advanced, won't touch network config instance_id = 978da889-c1eb-4016-b182-d8c4346a3c1d
992024-02-04T18:30:43.198ZINFOnexus: instance and vmm updated by sled agent file = nexus/src/app/instance.rs:1608 instance_id = 978da889-c1eb-4016-b182-d8c4346a3c1d instance_updated = false propolis_id = a99c950c-4d50-4241-8074-58986809f024 vmm_updated = true
1002024-02-04T18:30:43.198ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 45718 local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:51916 req_id = 84670375-f062-425e-9907-f41898766537 response_code = 204 uri = /instances/978da889-c1eb-4016-b182-d8c4346a3c1d
1012024-02-04T18:30:43.199ZINFOnexus: received new runtime state from sled agent file = nexus/src/app/instance.rs:1472 instance_id = 978da889-c1eb-4016-b182-d8c4346a3c1d instance_state = InstanceRuntimeState { propolis_id: Some(a99c950c-4d50-4241-8074-58986809f024), dst_propolis_id: None, migration_id: None, gen: Generation(3), time_updated: 2024-02-04T18:30:35.410980497Z } propolis_id = a99c950c-4d50-4241-8074-58986809f024 vmm_state = VmmRuntimeState { state: Running, gen: Generation(3), time_updated: 2024-02-04T18:30:43.198957364Z }
1022024-02-04T18:30:43.199ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:51916 req_id = e3a6b585-9f46-48bf-8889-3565c344a78c roles = RoleSet { roles: {} } uri = /instances/978da889-c1eb-4016-b182-d8c4346a3c1d
1032024-02-04T18:30:43.199ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:51916 req_id = e3a6b585-9f46-48bf-8889-3565c344a78c resource = Database result = Ok(()) uri = /instances/978da889-c1eb-4016-b182-d8c4346a3c1d
1042024-02-04T18:30:43.201ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:51916 req_id = e3a6b585-9f46-48bf-8889-3565c344a78c roles = RoleSet { roles: {} } uri = /instances/978da889-c1eb-4016-b182-d8c4346a3c1d
1052024-02-04T18:30:43.201ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:51916 req_id = e3a6b585-9f46-48bf-8889-3565c344a78c resource = Database result = Ok(()) uri = /instances/978da889-c1eb-4016-b182-d8c4346a3c1d
1062024-02-04T18:30:43.202ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:51916 req_id = e3a6b585-9f46-48bf-8889-3565c344a78c roles = RoleSet { roles: {} } uri = /instances/978da889-c1eb-4016-b182-d8c4346a3c1d
1072024-02-04T18:30:43.203ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:51916 req_id = e3a6b585-9f46-48bf-8889-3565c344a78c resource = Database result = Ok(()) uri = /instances/978da889-c1eb-4016-b182-d8c4346a3c1d
1082024-02-04T18:30:43.204ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:51916 req_id = e3a6b585-9f46-48bf-8889-3565c344a78c roles = RoleSet { roles: {} } uri = /instances/978da889-c1eb-4016-b182-d8c4346a3c1d
1092024-02-04T18:30:43.204ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:51916 req_id = e3a6b585-9f46-48bf-8889-3565c344a78c resource = Database result = Ok(()) uri = /instances/978da889-c1eb-4016-b182-d8c4346a3c1d
1102024-02-04T18:30:43.205ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:51916 req_id = e3a6b585-9f46-48bf-8889-3565c344a78c roles = RoleSet { roles: {} } uri = /instances/978da889-c1eb-4016-b182-d8c4346a3c1d
1112024-02-04T18:30:43.206ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:51916 req_id = e3a6b585-9f46-48bf-8889-3565c344a78c resource = Database result = Ok(()) uri = /instances/978da889-c1eb-4016-b182-d8c4346a3c1d
1122024-02-04T18:30:43.207ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:51916 req_id = e3a6b585-9f46-48bf-8889-3565c344a78c roles = RoleSet { roles: {} } uri = /instances/978da889-c1eb-4016-b182-d8c4346a3c1d
1132024-02-04T18:30:43.207ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:51916 req_id = e3a6b585-9f46-48bf-8889-3565c344a78c resource = Database result = Ok(()) uri = /instances/978da889-c1eb-4016-b182-d8c4346a3c1d
1142024-02-04T18:30:43.208ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:51916 req_id = e3a6b585-9f46-48bf-8889-3565c344a78c roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } uri = /instances/978da889-c1eb-4016-b182-d8c4346a3c1d
1152024-02-04T18:30:43.211ZDEBGnexus (dropshot_internal): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:51916 req_id = e3a6b585-9f46-48bf-8889-3565c344a78c resource = Instance { parent: Project { parent: Silo { parent: Fleet, key: 7eaca1c5-9568-4aa5-a1b2-cb905947fbfb, lookup_type: ById(7eaca1c5-9568-4aa5-a1b2-cb905947fbfb) }, key: 638a1648-cc73-4192-b4a6-cacb5faecf9d, lookup_type: ById(638a1648-cc73-4192-b4a6-cacb5faecf9d) }, key: 978da889-c1eb-4016-b182-d8c4346a3c1d, lookup_type: ById(978da889-c1eb-4016-b182-d8c4346a3c1d) } result = Ok(()) uri = /instances/978da889-c1eb-4016-b182-d8c4346a3c1d
1162024-02-04T18:30:43.212ZDEBGnexus: instance state generation already advanced, won't touch network config instance_id = 978da889-c1eb-4016-b182-d8c4346a3c1d
1172024-02-04T18:30:43.219ZINFOnexus: instance and vmm updated by sled agent file = nexus/src/app/instance.rs:1608 instance_id = 978da889-c1eb-4016-b182-d8c4346a3c1d instance_updated = false propolis_id = a99c950c-4d50-4241-8074-58986809f024 vmm_updated = true
1182024-02-04T18:30:43.219ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 20830 local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:51916 req_id = e3a6b585-9f46-48bf-8889-3565c344a78c response_code = 204 uri = /instances/978da889-c1eb-4016-b182-d8c4346a3c1d
1192024-02-04T18:30:44.019ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 1600 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:47021 req_id = 61f315a6-dd3b-4cdf-b90d-2bb9bf2b1d41 response_code = 200 uri = /metrics/collect/092d8721-ad83-4ea8-800d-5dfcaa02a610
1202024-02-04T18:30:45.763ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 43 reason = Timeout
1212024-02-04T18:30:45.763ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
1222024-02-04T18:30:45.763ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
1232024-02-04T18:30:45.765ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1242024-02-04T18:30:45.768ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
1252024-02-04T18:30:45.768ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
1262024-02-04T18:30:45.768ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
1272024-02-04T18:30:45.770ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 6.592983ms iteration = 43
1282024-02-04T18:30:54.019ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 1708 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:47021 req_id = d8026583-b4ba-42ab-ab60-8f4b41aece65 response_code = 200 uri = /metrics/collect/092d8721-ad83-4ea8-800d-5dfcaa02a610
1292024-02-04T18:30:55.751ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 10 reason = Timeout
1302024-02-04T18:30:55.751ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
1312024-02-04T18:30:55.751ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 10 reason = Timeout
1322024-02-04T18:30:55.751ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 10 reason = Timeout
1332024-02-04T18:30:55.751ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
1342024-02-04T18:30:55.751ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
1352024-02-04T18:30:55.751ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
1362024-02-04T18:30:55.751ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_internal dns_group = internal roles = RoleSet { roles: {} }
1372024-02-04T18:30:55.751ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 10 reason = Timeout
1382024-02-04T18:30:55.751ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 10 reason = Timeout
1392024-02-04T18:30:55.751ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
1402024-02-04T18:30:55.751ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_external dns_group = external roles = RoleSet { roles: {} }
1412024-02-04T18:30:55.752ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_internal dns_group = internal resource = Database result = Ok(())
1422024-02-04T18:30:55.752ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
1432024-02-04T18:30:55.752ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
1442024-02-04T18:30:55.752ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
1452024-02-04T18:30:55.752ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_external dns_group = external resource = Database result = Ok(())
1462024-02-04T18:30:55.752ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
1472024-02-04T18:30:55.752ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 16 reason = Timeout
1482024-02-04T18:30:55.752ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
1492024-02-04T18:30:55.753ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
1502024-02-04T18:30:55.753ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
1512024-02-04T18:30:55.753ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1522024-02-04T18:30:55.753ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1532024-02-04T18:30:55.754ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
1542024-02-04T18:30:55.754ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1552024-02-04T18:30:55.754ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
1562024-02-04T18:30:55.754ZDEBGnexus (DpdClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
1572024-02-04T18:30:55.754ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1582024-02-04T18:30:55.755ZDEBGnexus (DpdClient): client response result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "f5fd808b-a2b6-4645-b845-ee3dea8c7ffb", "content-length": "2", "date": "Sun, 04 Feb 2024 18:30:55 GMT"} })
1592024-02-04T18:30:55.755ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
1602024-02-04T18:30:55.755ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
1612024-02-04T18:30:55.755ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1622024-02-04T18:30:55.756ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 3.579505ms iteration = 16
1632024-02-04T18:30:55.757ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_external dns_group = external resource = Fleet result = Ok(())
1642024-02-04T18:30:55.757ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_external dns_group = external roles = RoleSet { roles: {} }
1652024-02-04T18:30:55.757ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_external dns_group = external resource = Database result = Ok(())
1662024-02-04T18:30:55.758ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_internal dns_group = internal resource = Fleet result = Ok(())
1672024-02-04T18:30:55.758ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_internal dns_group = internal roles = RoleSet { roles: {} }
1682024-02-04T18:30:55.758ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } dns_group = external
1692024-02-04T18:30:55.758ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 6.759045ms iteration = 10
1702024-02-04T18:30:55.758ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_internal dns_group = internal resource = Database result = Ok(())
1712024-02-04T18:30:55.758ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
1722024-02-04T18:30:55.759ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
1732024-02-04T18:30:55.759ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
1742024-02-04T18:30:55.759ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
1752024-02-04T18:30:55.759ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } dns_group = internal
1762024-02-04T18:30:55.759ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 8.002851ms iteration = 10
1772024-02-04T18:30:55.759ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
1782024-02-04T18:30:55.759ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
1792024-02-04T18:30:55.759ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
1802024-02-04T18:30:55.759ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
1812024-02-04T18:30:55.760ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
1822024-02-04T18:30:55.760ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
1832024-02-04T18:30:55.760ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
1842024-02-04T18:30:55.760ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
1852024-02-04T18:30:55.760ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
1862024-02-04T18:30:55.760ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
1872024-02-04T18:30:55.760ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
1882024-02-04T18:30:55.761ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
1892024-02-04T18:30:55.761ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1902024-02-04T18:30:55.761ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
1912024-02-04T18:30:55.763ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1922024-02-04T18:30:55.763ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1932024-02-04T18:30:55.763ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
1942024-02-04T18:30:55.763ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
1952024-02-04T18:30:55.763ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
1962024-02-04T18:30:55.764ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 44 reason = Timeout
1972024-02-04T18:30:55.764ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 12 reason = Timeout
1982024-02-04T18:30:55.764ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
1992024-02-04T18:30:55.764ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 9 reason = Timeout
2002024-02-04T18:30:55.764ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/blueprint_execution.rs:46 reason = no blueprint
2012024-02-04T18:30:55.764ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 4.96\u{b5}s iteration = 9
2022024-02-04T18:30:55.764ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 12 reason = Timeout
2032024-02-04T18:30:55.764ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 16 reason = Timeout
2042024-02-04T18:30:55.764ZWARNnexus (BackgroundTasks): phantom disk task started background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:46
2052024-02-04T18:30:55.764ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 16 reason = Timeout
2062024-02-04T18:30:55.764ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 16 reason = Timeout
2072024-02-04T18:30:55.764ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
2082024-02-04T18:30:55.764ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
2092024-02-04T18:30:55.764ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
2102024-02-04T18:30:55.764ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
2112024-02-04T18:30:55.764ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
2122024-02-04T18:30:55.765ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
2132024-02-04T18:30:55.765ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 64c3c424-e322-4f8a-b3f5-aea263693dd8 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
2142024-02-04T18:30:55.765ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
2152024-02-04T18:30:55.765ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
2162024-02-04T18:30:55.766ZDEBGnexus: client request MgAdminClient = [fd00:1122:3344:101::2]:4676 body = None method = GET uri = http://[fd00:1122:3344:101::2]:4676/bfd/peers
2172024-02-04T18:30:55.766ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2182024-02-04T18:30:55.766ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2192024-02-04T18:30:55.766ZDEBGnexus: client response MgAdminClient = [fd00:1122:3344:101::2]:4676 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(4676), path: "/bfd/peers", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "200ed08f-fe9f-4a46-bfcf-73501891abe3", "content-length": "2", "date": "Sun, 04 Feb 2024 18:30:55 GMT"} })
2202024-02-04T18:30:55.766ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 1.934454ms iteration = 16
2212024-02-04T18:30:55.766ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2222024-02-04T18:30:55.767ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
2232024-02-04T18:30:55.767ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
2242024-02-04T18:30:55.767ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
2252024-02-04T18:30:55.767ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
2262024-02-04T18:30:55.767ZWARNnexus (BackgroundTasks): phantom disk task done background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:92
2272024-02-04T18:30:55.767ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 2.905176ms iteration = 16
2282024-02-04T18:30:55.767ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
2292024-02-04T18:30:55.767ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
2302024-02-04T18:30:55.768ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
2312024-02-04T18:30:55.769ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 569ffb23-fbba-49ec-b235-2603da0b3eed dns_zone_name = oxide.test found_so_far = 0 version = 2
2322024-02-04T18:30:55.769ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
2332024-02-04T18:30:55.769ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
2342024-02-04T18:30:55.769ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
2352024-02-04T18:30:55.769ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
2362024-02-04T18:30:55.770ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
2372024-02-04T18:30:55.770ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
2382024-02-04T18:30:55.770ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
2392024-02-04T18:30:55.770ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2402024-02-04T18:30:55.770ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
2412024-02-04T18:30:55.770ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
2422024-02-04T18:30:55.770ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2432024-02-04T18:30:55.770ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
2442024-02-04T18:30:55.771ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 7.220011ms iteration = 44
2452024-02-04T18:30:55.772ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
2462024-02-04T18:30:55.772ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
2472024-02-04T18:30:55.772ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
2482024-02-04T18:30:55.773ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
2492024-02-04T18:30:55.773ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
2502024-02-04T18:30:55.773ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
2512024-02-04T18:30:55.773ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
2522024-02-04T18:30:55.773ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
2532024-02-04T18:30:55.774ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
2542024-02-04T18:30:55.774ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
2552024-02-04T18:30:55.774ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
2562024-02-04T18:30:55.775ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 64c3c424-e322-4f8a-b3f5-aea263693dd8 dns_zone_name = control-plane.oxide.internal found_so_far = 51 version = 1
2572024-02-04T18:30:55.775ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
2582024-02-04T18:30:55.775ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2024-02-04 18:24:12.000370 UTC dns_group = internal generation = 1
2592024-02-04T18:30:55.775ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 23.612488ms iteration = 10
2602024-02-04T18:30:55.775ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 569ffb23-fbba-49ec-b235-2603da0b3eed dns_zone_name = oxide.test found_so_far = 1 version = 2
2612024-02-04T18:30:55.775ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
2622024-02-04T18:30:55.775ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2024-02-04 18:24:12.376697 UTC dns_group = external generation = 2
2632024-02-04T18:30:55.775ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 24.142215ms iteration = 10
2642024-02-04T18:30:55.782ZINFOnexus (BackgroundTasks): found Silo/DNS/TLS config (changed) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2024-02-04T18:23:55.840039Z, time_modified: 2024-02-04T18:23:55.840039Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 7eaca1c5-9568-4aa5-a1b2-cb905947fbfb, db_silo: Silo { identity: SiloIdentity { id: 7eaca1c5-9568-4aa5-a1b2-cb905947fbfb, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-02-04T18:24:12.352343Z, time_modified: 2024-02-04T18:24:12.352343Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "209b27934f4c6206a8d147070bebbcc41c8953fdb7a37a8a7847a104d18adc4c" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 7eaca1c5-9568-4aa5-a1b2-cb905947fbfb, db_silo: Silo { identity: SiloIdentity { id: 7eaca1c5-9568-4aa5-a1b2-cb905947fbfb, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-02-04T18:24:12.352343Z, time_modified: 2024-02-04T18:24:12.352343Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "209b27934f4c6206a8d147070bebbcc41c8953fdb7a37a8a7847a104d18adc4c" }] }) } file = nexus/src/app/background/external_endpoints.rs:101
2652024-02-04T18:30:55.782ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 30.955202ms iteration = 10
2662024-02-04T18:30:55.783ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
2672024-02-04T18:30:55.783ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
2682024-02-04T18:30:55.784ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
2692024-02-04T18:30:55.784ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
2702024-02-04T18:30:55.785ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2712024-02-04T18:30:55.788ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 339b514e-8533-421a-a5ed-83b7bd206824, lookup_type: ById(339b514e-8533-421a-a5ed-83b7bd206824) } result = Ok(())
2722024-02-04T18:30:55.788ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
2732024-02-04T18:30:55.788ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
2742024-02-04T18:30:55.789ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 24.764373ms iteration = 16
2752024-02-04T18:30:55.833ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
2762024-02-04T18:30:55.837ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
2772024-02-04T18:30:55.894ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
2782024-02-04T18:30:55.894ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::8)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "9f1d2ce9-29e7-40f4-b98f-8fa3e5a056fe", "date": "Sun, 04 Feb 2024 18:30:55 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
2792024-02-04T18:30:55.894ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
2802024-02-04T18:30:55.895ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::9)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "a6894b21-e723-42a8-a6e8-cfcc1791dcca", "date": "Sun, 04 Feb 2024 18:30:55 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
2812024-02-04T18:30:55.895ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
2822024-02-04T18:30:55.895ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
2832024-02-04T18:30:55.895ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 131.316308ms iteration = 12
2842024-02-04T18:30:55.896ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
2852024-02-04T18:30:55.927ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
2862024-02-04T18:30:55.928ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:2::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "0f686021-ffba-425d-acdd-ec7d7a2bae6e", "date": "Sun, 04 Feb 2024 18:30:55 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
2872024-02-04T18:30:55.928ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
2882024-02-04T18:30:55.929ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:1::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "948f6caf-2925-41d1-88e7-5d6bb5725117", "date": "Sun, 04 Feb 2024 18:30:55 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
2892024-02-04T18:30:55.929ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
2902024-02-04T18:30:55.930ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:3::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "97c32e15-2745-45d9-a07a-c7ee0bd8c4cd", "date": "Sun, 04 Feb 2024 18:30:55 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
2912024-02-04T18:30:55.930ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
2922024-02-04T18:30:55.932ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
2932024-02-04T18:30:55.932ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 168.406918ms iteration = 12
2942024-02-04T18:31:00.441ZINFOnexus: received new runtime state from sled agent file = nexus/src/app/instance.rs:1472 instance_id = 978da889-c1eb-4016-b182-d8c4346a3c1d instance_state = InstanceRuntimeState { propolis_id: Some(a99c950c-4d50-4241-8074-58986809f024), dst_propolis_id: None, migration_id: None, gen: Generation(3), time_updated: 2024-02-04T18:30:35.410980497Z } propolis_id = a99c950c-4d50-4241-8074-58986809f024 vmm_state = VmmRuntimeState { state: Stopping, gen: Generation(5), time_updated: 2024-02-04T18:31:00.441503679Z }
2952024-02-04T18:31:00.441ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:51916 req_id = 10f99280-cdf7-4c09-8187-7de893005a50 roles = RoleSet { roles: {} } uri = /instances/978da889-c1eb-4016-b182-d8c4346a3c1d
2962024-02-04T18:31:00.442ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:51916 req_id = 10f99280-cdf7-4c09-8187-7de893005a50 resource = Database result = Ok(()) uri = /instances/978da889-c1eb-4016-b182-d8c4346a3c1d
2972024-02-04T18:31:00.443ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:51916 req_id = 10f99280-cdf7-4c09-8187-7de893005a50 roles = RoleSet { roles: {} } uri = /instances/978da889-c1eb-4016-b182-d8c4346a3c1d
2982024-02-04T18:31:00.444ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:51916 req_id = 10f99280-cdf7-4c09-8187-7de893005a50 resource = Database result = Ok(()) uri = /instances/978da889-c1eb-4016-b182-d8c4346a3c1d
2992024-02-04T18:31:00.445ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:51916 req_id = 10f99280-cdf7-4c09-8187-7de893005a50 roles = RoleSet { roles: {} } uri = /instances/978da889-c1eb-4016-b182-d8c4346a3c1d
3002024-02-04T18:31:00.445ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:51916 req_id = 10f99280-cdf7-4c09-8187-7de893005a50 resource = Database result = Ok(()) uri = /instances/978da889-c1eb-4016-b182-d8c4346a3c1d
3012024-02-04T18:31:00.446ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:51916 req_id = 10f99280-cdf7-4c09-8187-7de893005a50 roles = RoleSet { roles: {} } uri = /instances/978da889-c1eb-4016-b182-d8c4346a3c1d
3022024-02-04T18:31:00.446ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:51916 req_id = 10f99280-cdf7-4c09-8187-7de893005a50 resource = Database result = Ok(()) uri = /instances/978da889-c1eb-4016-b182-d8c4346a3c1d
3032024-02-04T18:31:00.448ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:51916 req_id = 10f99280-cdf7-4c09-8187-7de893005a50 roles = RoleSet { roles: {} } uri = /instances/978da889-c1eb-4016-b182-d8c4346a3c1d
3042024-02-04T18:31:00.448ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:51916 req_id = 10f99280-cdf7-4c09-8187-7de893005a50 resource = Database result = Ok(()) uri = /instances/978da889-c1eb-4016-b182-d8c4346a3c1d
3052024-02-04T18:31:00.449ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:51916 req_id = 10f99280-cdf7-4c09-8187-7de893005a50 roles = RoleSet { roles: {} } uri = /instances/978da889-c1eb-4016-b182-d8c4346a3c1d
3062024-02-04T18:31:00.449ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:51916 req_id = 10f99280-cdf7-4c09-8187-7de893005a50 resource = Database result = Ok(()) uri = /instances/978da889-c1eb-4016-b182-d8c4346a3c1d
3072024-02-04T18:31:00.450ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:51916 req_id = 10f99280-cdf7-4c09-8187-7de893005a50 roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } uri = /instances/978da889-c1eb-4016-b182-d8c4346a3c1d
3082024-02-04T18:31:00.453ZDEBGnexus (dropshot_internal): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:51916 req_id = 10f99280-cdf7-4c09-8187-7de893005a50 resource = Instance { parent: Project { parent: Silo { parent: Fleet, key: 7eaca1c5-9568-4aa5-a1b2-cb905947fbfb, lookup_type: ById(7eaca1c5-9568-4aa5-a1b2-cb905947fbfb) }, key: 638a1648-cc73-4192-b4a6-cacb5faecf9d, lookup_type: ById(638a1648-cc73-4192-b4a6-cacb5faecf9d) }, key: 978da889-c1eb-4016-b182-d8c4346a3c1d, lookup_type: ById(978da889-c1eb-4016-b182-d8c4346a3c1d) } result = Ok(()) uri = /instances/978da889-c1eb-4016-b182-d8c4346a3c1d
3092024-02-04T18:31:00.453ZDEBGnexus: instance state generation already advanced, won't touch network config instance_id = 978da889-c1eb-4016-b182-d8c4346a3c1d
3102024-02-04T18:31:00.461ZINFOnexus: instance and vmm updated by sled agent file = nexus/src/app/instance.rs:1608 instance_id = 978da889-c1eb-4016-b182-d8c4346a3c1d instance_updated = false propolis_id = a99c950c-4d50-4241-8074-58986809f024 vmm_updated = true
3112024-02-04T18:31:00.461ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 20114 local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:51916 req_id = 10f99280-cdf7-4c09-8187-7de893005a50 response_code = 204 uri = /instances/978da889-c1eb-4016-b182-d8c4346a3c1d
3122024-02-04T18:31:00.462ZINFOnexus: received new runtime state from sled agent file = nexus/src/app/instance.rs:1472 instance_id = 978da889-c1eb-4016-b182-d8c4346a3c1d instance_state = InstanceRuntimeState { propolis_id: Some(a99c950c-4d50-4241-8074-58986809f024), dst_propolis_id: None, migration_id: None, gen: Generation(3), time_updated: 2024-02-04T18:30:35.410980497Z } propolis_id = a99c950c-4d50-4241-8074-58986809f024 vmm_state = VmmRuntimeState { state: Stopping, gen: Generation(6), time_updated: 2024-02-04T18:31:00.462043958Z }
3132024-02-04T18:31:00.462ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:51916 req_id = 406306ee-c6ba-4521-80c3-22fbdfc51b79 roles = RoleSet { roles: {} } uri = /instances/978da889-c1eb-4016-b182-d8c4346a3c1d
3142024-02-04T18:31:00.462ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:51916 req_id = 406306ee-c6ba-4521-80c3-22fbdfc51b79 resource = Database result = Ok(()) uri = /instances/978da889-c1eb-4016-b182-d8c4346a3c1d
3152024-02-04T18:31:00.463ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:51916 req_id = 406306ee-c6ba-4521-80c3-22fbdfc51b79 roles = RoleSet { roles: {} } uri = /instances/978da889-c1eb-4016-b182-d8c4346a3c1d
3162024-02-04T18:31:00.463ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:51916 req_id = 406306ee-c6ba-4521-80c3-22fbdfc51b79 resource = Database result = Ok(()) uri = /instances/978da889-c1eb-4016-b182-d8c4346a3c1d
3172024-02-04T18:31:00.464ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:51916 req_id = 406306ee-c6ba-4521-80c3-22fbdfc51b79 roles = RoleSet { roles: {} } uri = /instances/978da889-c1eb-4016-b182-d8c4346a3c1d
3182024-02-04T18:31:00.464ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:51916 req_id = 406306ee-c6ba-4521-80c3-22fbdfc51b79 resource = Database result = Ok(()) uri = /instances/978da889-c1eb-4016-b182-d8c4346a3c1d
3192024-02-04T18:31:00.465ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:51916 req_id = 406306ee-c6ba-4521-80c3-22fbdfc51b79 roles = RoleSet { roles: {} } uri = /instances/978da889-c1eb-4016-b182-d8c4346a3c1d
3202024-02-04T18:31:00.466ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:51916 req_id = 406306ee-c6ba-4521-80c3-22fbdfc51b79 resource = Database result = Ok(()) uri = /instances/978da889-c1eb-4016-b182-d8c4346a3c1d
3212024-02-04T18:31:00.466ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:51916 req_id = 406306ee-c6ba-4521-80c3-22fbdfc51b79 roles = RoleSet { roles: {} } uri = /instances/978da889-c1eb-4016-b182-d8c4346a3c1d
3222024-02-04T18:31:00.467ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:51916 req_id = 406306ee-c6ba-4521-80c3-22fbdfc51b79 resource = Database result = Ok(()) uri = /instances/978da889-c1eb-4016-b182-d8c4346a3c1d
3232024-02-04T18:31:00.468ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:51916 req_id = 406306ee-c6ba-4521-80c3-22fbdfc51b79 roles = RoleSet { roles: {} } uri = /instances/978da889-c1eb-4016-b182-d8c4346a3c1d
3242024-02-04T18:31:00.468ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:51916 req_id = 406306ee-c6ba-4521-80c3-22fbdfc51b79 resource = Database result = Ok(()) uri = /instances/978da889-c1eb-4016-b182-d8c4346a3c1d
3252024-02-04T18:31:00.469ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:51916 req_id = 406306ee-c6ba-4521-80c3-22fbdfc51b79 roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } uri = /instances/978da889-c1eb-4016-b182-d8c4346a3c1d
3262024-02-04T18:31:00.472ZDEBGnexus (dropshot_internal): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:51916 req_id = 406306ee-c6ba-4521-80c3-22fbdfc51b79 resource = Instance { parent: Project { parent: Silo { parent: Fleet, key: 7eaca1c5-9568-4aa5-a1b2-cb905947fbfb, lookup_type: ById(7eaca1c5-9568-4aa5-a1b2-cb905947fbfb) }, key: 638a1648-cc73-4192-b4a6-cacb5faecf9d, lookup_type: ById(638a1648-cc73-4192-b4a6-cacb5faecf9d) }, key: 978da889-c1eb-4016-b182-d8c4346a3c1d, lookup_type: ById(978da889-c1eb-4016-b182-d8c4346a3c1d) } result = Ok(()) uri = /instances/978da889-c1eb-4016-b182-d8c4346a3c1d
3272024-02-04T18:31:00.472ZDEBGnexus: instance state generation already advanced, won't touch network config instance_id = 978da889-c1eb-4016-b182-d8c4346a3c1d
3282024-02-04T18:31:00.479ZINFOnexus: instance and vmm updated by sled agent file = nexus/src/app/instance.rs:1608 instance_id = 978da889-c1eb-4016-b182-d8c4346a3c1d instance_updated = false propolis_id = a99c950c-4d50-4241-8074-58986809f024 vmm_updated = true
3292024-02-04T18:31:00.479ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 17696 local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:51916 req_id = 406306ee-c6ba-4521-80c3-22fbdfc51b79 response_code = 204 uri = /instances/978da889-c1eb-4016-b182-d8c4346a3c1d
3302024-02-04T18:31:01.222ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:44921 req_id = 563a3aa4-06ea-4a5d-8b90-bc7fbc049b2a roles = RoleSet { roles: {} } uri = /nat/ipv4/changeset/14?limit=100
3312024-02-04T18:31:01.222ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:44921 req_id = 563a3aa4-06ea-4a5d-8b90-bc7fbc049b2a resource = Database result = Ok(()) uri = /nat/ipv4/changeset/14?limit=100
3322024-02-04T18:31:01.225ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 2835 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:44921 req_id = 563a3aa4-06ea-4a5d-8b90-bc7fbc049b2a response_code = 200 uri = /nat/ipv4/changeset/14?limit=100
3332024-02-04T18:31:01.226ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:44921 req_id = 0dc857ec-5ccc-4338-b5cf-163e2fa5232f roles = RoleSet { roles: {} } uri = /nat/ipv4/changeset/16?limit=100
3342024-02-04T18:31:01.226ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:44921 req_id = 0dc857ec-5ccc-4338-b5cf-163e2fa5232f resource = Database result = Ok(()) uri = /nat/ipv4/changeset/16?limit=100
3352024-02-04T18:31:01.227ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 1415 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:44921 req_id = 0dc857ec-5ccc-4338-b5cf-163e2fa5232f response_code = 200 uri = /nat/ipv4/changeset/16?limit=100
3362024-02-04T18:31:02.228ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:44921 req_id = 6c727e0e-34bb-4997-a280-19ac872cbd46 roles = RoleSet { roles: {} } uri = /nat/ipv4/changeset/16?limit=100
3372024-02-04T18:31:02.229ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:44921 req_id = 6c727e0e-34bb-4997-a280-19ac872cbd46 resource = Database result = Ok(()) uri = /nat/ipv4/changeset/16?limit=100
3382024-02-04T18:31:02.231ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 2141 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:44921 req_id = 6c727e0e-34bb-4997-a280-19ac872cbd46 response_code = 200 uri = /nat/ipv4/changeset/16?limit=100
3392024-02-04T18:31:02.782ZINFOnexus: received new runtime state from sled agent file = nexus/src/app/instance.rs:1472 instance_id = 978da889-c1eb-4016-b182-d8c4346a3c1d instance_state = InstanceRuntimeState { propolis_id: None, dst_propolis_id: None, migration_id: None, gen: Generation(5), time_updated: 2024-02-04T18:31:00.640855182Z } propolis_id = a99c950c-4d50-4241-8074-58986809f024 vmm_state = VmmRuntimeState { state: Destroyed, gen: Generation(7), time_updated: 2024-02-04T18:31:00.640855182Z }
3402024-02-04T18:31:02.782ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:51916 req_id = 063dfd49-22df-4e7e-914d-ec2d8424b16d roles = RoleSet { roles: {} } uri = /instances/978da889-c1eb-4016-b182-d8c4346a3c1d
3412024-02-04T18:31:02.783ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:51916 req_id = 063dfd49-22df-4e7e-914d-ec2d8424b16d resource = Database result = Ok(()) uri = /instances/978da889-c1eb-4016-b182-d8c4346a3c1d
3422024-02-04T18:31:02.784ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:51916 req_id = 063dfd49-22df-4e7e-914d-ec2d8424b16d roles = RoleSet { roles: {} } uri = /instances/978da889-c1eb-4016-b182-d8c4346a3c1d
3432024-02-04T18:31:02.784ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:51916 req_id = 063dfd49-22df-4e7e-914d-ec2d8424b16d resource = Database result = Ok(()) uri = /instances/978da889-c1eb-4016-b182-d8c4346a3c1d
3442024-02-04T18:31:02.786ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:51916 req_id = 063dfd49-22df-4e7e-914d-ec2d8424b16d roles = RoleSet { roles: {} } uri = /instances/978da889-c1eb-4016-b182-d8c4346a3c1d
3452024-02-04T18:31:02.786ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:51916 req_id = 063dfd49-22df-4e7e-914d-ec2d8424b16d resource = Database result = Ok(()) uri = /instances/978da889-c1eb-4016-b182-d8c4346a3c1d
3462024-02-04T18:31:02.787ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:51916 req_id = 063dfd49-22df-4e7e-914d-ec2d8424b16d roles = RoleSet { roles: {} } uri = /instances/978da889-c1eb-4016-b182-d8c4346a3c1d
3472024-02-04T18:31:02.787ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:51916 req_id = 063dfd49-22df-4e7e-914d-ec2d8424b16d resource = Database result = Ok(()) uri = /instances/978da889-c1eb-4016-b182-d8c4346a3c1d
3482024-02-04T18:31:02.789ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:51916 req_id = 063dfd49-22df-4e7e-914d-ec2d8424b16d roles = RoleSet { roles: {} } uri = /instances/978da889-c1eb-4016-b182-d8c4346a3c1d
3492024-02-04T18:31:02.789ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:51916 req_id = 063dfd49-22df-4e7e-914d-ec2d8424b16d resource = Database result = Ok(()) uri = /instances/978da889-c1eb-4016-b182-d8c4346a3c1d
3502024-02-04T18:31:02.790ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:51916 req_id = 063dfd49-22df-4e7e-914d-ec2d8424b16d roles = RoleSet { roles: {} } uri = /instances/978da889-c1eb-4016-b182-d8c4346a3c1d
3512024-02-04T18:31:02.790ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:51916 req_id = 063dfd49-22df-4e7e-914d-ec2d8424b16d resource = Database result = Ok(()) uri = /instances/978da889-c1eb-4016-b182-d8c4346a3c1d
3522024-02-04T18:31:02.791ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:51916 req_id = 063dfd49-22df-4e7e-914d-ec2d8424b16d roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } uri = /instances/978da889-c1eb-4016-b182-d8c4346a3c1d
3532024-02-04T18:31:02.794ZDEBGnexus (dropshot_internal): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:51916 req_id = 063dfd49-22df-4e7e-914d-ec2d8424b16d resource = Instance { parent: Project { parent: Silo { parent: Fleet, key: 7eaca1c5-9568-4aa5-a1b2-cb905947fbfb, lookup_type: ById(7eaca1c5-9568-4aa5-a1b2-cb905947fbfb) }, key: 638a1648-cc73-4192-b4a6-cacb5faecf9d, lookup_type: ById(638a1648-cc73-4192-b4a6-cacb5faecf9d) }, key: 978da889-c1eb-4016-b182-d8c4346a3c1d, lookup_type: ById(978da889-c1eb-4016-b182-d8c4346a3c1d) } result = Ok(()) uri = /instances/978da889-c1eb-4016-b182-d8c4346a3c1d
3542024-02-04T18:31:02.794ZINFOnexus: instance cleared its Propolis ID, cleaning network config file = nexus/src/app/instance_network.rs:739 instance_id = 978da889-c1eb-4016-b182-d8c4346a3c1d propolis_id = Some(a99c950c-4d50-4241-8074-58986809f024)
3552024-02-04T18:31:02.794ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:51916 req_id = 063dfd49-22df-4e7e-914d-ec2d8424b16d roles = RoleSet { roles: {} } uri = /instances/978da889-c1eb-4016-b182-d8c4346a3c1d
3562024-02-04T18:31:02.794ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:51916 req_id = 063dfd49-22df-4e7e-914d-ec2d8424b16d resource = Database result = Ok(()) uri = /instances/978da889-c1eb-4016-b182-d8c4346a3c1d
3572024-02-04T18:31:02.796ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:51916 req_id = 063dfd49-22df-4e7e-914d-ec2d8424b16d roles = RoleSet { roles: {} } uri = /instances/978da889-c1eb-4016-b182-d8c4346a3c1d
3582024-02-04T18:31:02.796ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:51916 req_id = 063dfd49-22df-4e7e-914d-ec2d8424b16d resource = Database result = Ok(()) uri = /instances/978da889-c1eb-4016-b182-d8c4346a3c1d
3592024-02-04T18:31:02.797ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:51916 req_id = 063dfd49-22df-4e7e-914d-ec2d8424b16d roles = RoleSet { roles: {} } uri = /instances/978da889-c1eb-4016-b182-d8c4346a3c1d
3602024-02-04T18:31:02.797ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:51916 req_id = 063dfd49-22df-4e7e-914d-ec2d8424b16d resource = Database result = Ok(()) uri = /instances/978da889-c1eb-4016-b182-d8c4346a3c1d
3612024-02-04T18:31:02.798ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:51916 req_id = 063dfd49-22df-4e7e-914d-ec2d8424b16d roles = RoleSet { roles: {} } uri = /instances/978da889-c1eb-4016-b182-d8c4346a3c1d
3622024-02-04T18:31:02.798ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:51916 req_id = 063dfd49-22df-4e7e-914d-ec2d8424b16d resource = Database result = Ok(()) uri = /instances/978da889-c1eb-4016-b182-d8c4346a3c1d
3632024-02-04T18:31:02.799ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:51916 req_id = 063dfd49-22df-4e7e-914d-ec2d8424b16d roles = RoleSet { roles: {} } uri = /instances/978da889-c1eb-4016-b182-d8c4346a3c1d
3642024-02-04T18:31:02.799ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:51916 req_id = 063dfd49-22df-4e7e-914d-ec2d8424b16d resource = Database result = Ok(()) uri = /instances/978da889-c1eb-4016-b182-d8c4346a3c1d
3652024-02-04T18:31:02.800ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:51916 req_id = 063dfd49-22df-4e7e-914d-ec2d8424b16d roles = RoleSet { roles: {} } uri = /instances/978da889-c1eb-4016-b182-d8c4346a3c1d
3662024-02-04T18:31:02.801ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:51916 req_id = 063dfd49-22df-4e7e-914d-ec2d8424b16d resource = Database result = Ok(()) uri = /instances/978da889-c1eb-4016-b182-d8c4346a3c1d
3672024-02-04T18:31:02.802ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:51916 req_id = 063dfd49-22df-4e7e-914d-ec2d8424b16d roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } uri = /instances/978da889-c1eb-4016-b182-d8c4346a3c1d
3682024-02-04T18:31:02.805ZDEBGnexus (dropshot_internal): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:51916 req_id = 063dfd49-22df-4e7e-914d-ec2d8424b16d resource = Instance { parent: Project { parent: Silo { parent: Fleet, key: 7eaca1c5-9568-4aa5-a1b2-cb905947fbfb, lookup_type: ById(7eaca1c5-9568-4aa5-a1b2-cb905947fbfb) }, key: 638a1648-cc73-4192-b4a6-cacb5faecf9d, lookup_type: ById(638a1648-cc73-4192-b4a6-cacb5faecf9d) }, key: 978da889-c1eb-4016-b182-d8c4346a3c1d, lookup_type: ById(978da889-c1eb-4016-b182-d8c4346a3c1d) } result = Ok(()) uri = /instances/978da889-c1eb-4016-b182-d8c4346a3c1d
3692024-02-04T18:31:02.805ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:51916 req_id = 063dfd49-22df-4e7e-914d-ec2d8424b16d roles = RoleSet { roles: {} } uri = /instances/978da889-c1eb-4016-b182-d8c4346a3c1d
3702024-02-04T18:31:02.805ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:51916 req_id = 063dfd49-22df-4e7e-914d-ec2d8424b16d resource = Database result = Ok(()) uri = /instances/978da889-c1eb-4016-b182-d8c4346a3c1d
3712024-02-04T18:31:02.806ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:51916 req_id = 063dfd49-22df-4e7e-914d-ec2d8424b16d roles = RoleSet { roles: {} } uri = /instances/978da889-c1eb-4016-b182-d8c4346a3c1d
3722024-02-04T18:31:02.806ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:51916 req_id = 063dfd49-22df-4e7e-914d-ec2d8424b16d resource = Database result = Ok(()) uri = /instances/978da889-c1eb-4016-b182-d8c4346a3c1d
3732024-02-04T18:31:02.807ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:51916 req_id = 063dfd49-22df-4e7e-914d-ec2d8424b16d roles = RoleSet { roles: {} } uri = /instances/978da889-c1eb-4016-b182-d8c4346a3c1d
3742024-02-04T18:31:02.807ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:51916 req_id = 063dfd49-22df-4e7e-914d-ec2d8424b16d resource = Database result = Ok(()) uri = /instances/978da889-c1eb-4016-b182-d8c4346a3c1d
3752024-02-04T18:31:02.808ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:51916 req_id = 063dfd49-22df-4e7e-914d-ec2d8424b16d roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } uri = /instances/978da889-c1eb-4016-b182-d8c4346a3c1d
3762024-02-04T18:31:02.811ZDEBGnexus (dropshot_internal): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:51916 req_id = 063dfd49-22df-4e7e-914d-ec2d8424b16d resource = Instance { parent: Project { parent: Silo { parent: Fleet, key: 7eaca1c5-9568-4aa5-a1b2-cb905947fbfb, lookup_type: ById(7eaca1c5-9568-4aa5-a1b2-cb905947fbfb) }, key: 638a1648-cc73-4192-b4a6-cacb5faecf9d, lookup_type: ById(638a1648-cc73-4192-b4a6-cacb5faecf9d) }, key: 978da889-c1eb-4016-b182-d8c4346a3c1d, lookup_type: ById(978da889-c1eb-4016-b182-d8c4346a3c1d) } result = Ok(()) uri = /instances/978da889-c1eb-4016-b182-d8c4346a3c1d
3772024-02-04T18:31:02.811ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:51916 req_id = 063dfd49-22df-4e7e-914d-ec2d8424b16d roles = RoleSet { roles: {} } uri = /instances/978da889-c1eb-4016-b182-d8c4346a3c1d
3782024-02-04T18:31:02.811ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:51916 req_id = 063dfd49-22df-4e7e-914d-ec2d8424b16d resource = Database result = Ok(()) uri = /instances/978da889-c1eb-4016-b182-d8c4346a3c1d
3792024-02-04T18:31:02.815ZDEBGnexus (InstanceAllocator): roles roles = RoleSet { roles: {} }
3802024-02-04T18:31:02.815ZDEBGnexus (InstanceAllocator): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Database result = Ok(())
3812024-02-04T18:31:02.816ZDEBGnexus (InstanceAllocator): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "viewer")} }
3822024-02-04T18:31:02.817ZDEBGnexus (InstanceAllocator): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Fleet result = Ok(())
3832024-02-04T18:31:02.817ZDEBGnexus (InstanceAllocator): roles roles = RoleSet { roles: {} }
3842024-02-04T18:31:02.817ZDEBGnexus (InstanceAllocator): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Database result = Ok(())
3852024-02-04T18:31:02.819ZDEBGnexus (InstanceAllocator): roles roles = RoleSet { roles: {} }
3862024-02-04T18:31:02.819ZDEBGnexus (InstanceAllocator): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Database result = Ok(())
3872024-02-04T18:31:02.820ZDEBGnexus (InstanceAllocator): roles roles = RoleSet { roles: {} }
3882024-02-04T18:31:02.820ZDEBGnexus (InstanceAllocator): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Database result = Ok(())
3892024-02-04T18:31:02.821ZDEBGnexus (InstanceAllocator): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "viewer")} }
3902024-02-04T18:31:02.822ZDEBGnexus (InstanceAllocator): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Sled { parent: Fleet, key: 339b514e-8533-421a-a5ed-83b7bd206824, lookup_type: ById(339b514e-8533-421a-a5ed-83b7bd206824) } result = Ok(())
3912024-02-04T18:31:02.855ZDEBGnexus: client request SledAgent = 339b514e-8533-421a-a5ed-83b7bd206824 body = Some(Body) method = DELETE uri = http://[fd00:1122:3344:101::1]:12345/v2p/a7cceed3-b50c-4c3b-871b-a14d7152a052
3922024-02-04T18:31:02.856ZDEBGnexus: client response SledAgent = 339b514e-8533-421a-a5ed-83b7bd206824 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/v2p/a7cceed3-b50c-4c3b-871b-a14d7152a052", query: None, fragment: None }, status: 204, headers: {"x-request-id": "88d020e7-ecc7-4739-8713-3a52a3801ced", "date": "Sun, 04 Feb 2024 18:31:02 GMT"} })
3932024-02-04T18:31:02.858ZINFOnexus: deleting instance dpd configuration file = nexus/src/app/instance_network.rs:519 instance_id = 978da889-c1eb-4016-b182-d8c4346a3c1d
3942024-02-04T18:31:02.858ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:51916 req_id = 063dfd49-22df-4e7e-914d-ec2d8424b16d roles = RoleSet { roles: {} } uri = /instances/978da889-c1eb-4016-b182-d8c4346a3c1d
3952024-02-04T18:31:02.858ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:51916 req_id = 063dfd49-22df-4e7e-914d-ec2d8424b16d resource = Database result = Ok(()) uri = /instances/978da889-c1eb-4016-b182-d8c4346a3c1d
3962024-02-04T18:31:02.860ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:51916 req_id = 063dfd49-22df-4e7e-914d-ec2d8424b16d roles = RoleSet { roles: {} } uri = /instances/978da889-c1eb-4016-b182-d8c4346a3c1d
3972024-02-04T18:31:02.860ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:51916 req_id = 063dfd49-22df-4e7e-914d-ec2d8424b16d resource = Database result = Ok(()) uri = /instances/978da889-c1eb-4016-b182-d8c4346a3c1d
3982024-02-04T18:31:02.862ZWARNnexus: no matching nat entries to soft delete file = nexus/src/app/instance_network.rs:609
3992024-02-04T18:31:02.862ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:51916 req_id = 063dfd49-22df-4e7e-914d-ec2d8424b16d roles = RoleSet { roles: {} } uri = /instances/978da889-c1eb-4016-b182-d8c4346a3c1d
4002024-02-04T18:31:02.862ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:51916 req_id = 063dfd49-22df-4e7e-914d-ec2d8424b16d resource = Database result = Ok(()) uri = /instances/978da889-c1eb-4016-b182-d8c4346a3c1d
4012024-02-04T18:31:02.863ZWARNnexus: no matching nat entries to soft delete file = nexus/src/app/instance_network.rs:609
4022024-02-04T18:31:02.863ZDEBGnexus (InstanceAllocator): roles roles = RoleSet { roles: {} }
4032024-02-04T18:31:02.863ZDEBGnexus (InstanceAllocator): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Database result = Ok(())
4042024-02-04T18:31:02.865ZDEBGnexus (InstanceAllocator): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "viewer")} }
4052024-02-04T18:31:02.866ZDEBGnexus (InstanceAllocator): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Fleet result = Ok(())
4062024-02-04T18:31:02.866ZDEBGnexus (InstanceAllocator): roles roles = RoleSet { roles: {} }
4072024-02-04T18:31:02.866ZDEBGnexus (InstanceAllocator): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Database result = Ok(())
4082024-02-04T18:31:02.869ZDEBGnexus: notifying dendrite of updates instance_id = Some(978da889-c1eb-4016-b182-d8c4346a3c1d) switch = switch0
4092024-02-04T18:31:02.869ZDEBGnexus (DpdClient): client request body = None method = POST uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/trigger
4102024-02-04T18:31:02.869ZDEBGnexus (DpdClient): client response result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/trigger", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "40750c0a-4f95-4bc9-ac0d-115b767c2449", "content-length": "4", "date": "Sun, 04 Feb 2024 18:31:02 GMT"} })
4112024-02-04T18:31:02.869ZDEBGnexus (InstanceAllocator): roles roles = RoleSet { roles: {} }
4122024-02-04T18:31:02.870ZDEBGnexus (InstanceAllocator): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Database result = Ok(())
4132024-02-04T18:31:02.871ZDEBGnexus (InstanceAllocator): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "viewer")} }
4142024-02-04T18:31:02.872ZDEBGnexus (InstanceAllocator): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Fleet result = Ok(())
4152024-02-04T18:31:02.872ZDEBGnexus (InstanceAllocator): roles roles = RoleSet { roles: {} }
4162024-02-04T18:31:02.872ZDEBGnexus (InstanceAllocator): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Database result = Ok(())
4172024-02-04T18:31:02.875ZDEBGnexus: notifying dendrite of updates instance_id = Some(978da889-c1eb-4016-b182-d8c4346a3c1d) switch = switch0
4182024-02-04T18:31:02.875ZDEBGnexus (DpdClient): client request body = None method = POST uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/trigger
4192024-02-04T18:31:02.875ZDEBGnexus (DpdClient): client response result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/trigger", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "7da61fd4-408b-40bf-8846-49c91fc293b9", "content-length": "4", "date": "Sun, 04 Feb 2024 18:31:02 GMT"} })
4202024-02-04T18:31:02.875ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:51916 req_id = 063dfd49-22df-4e7e-914d-ec2d8424b16d roles = RoleSet { roles: {} } uri = /instances/978da889-c1eb-4016-b182-d8c4346a3c1d
4212024-02-04T18:31:02.875ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:51916 req_id = 063dfd49-22df-4e7e-914d-ec2d8424b16d resource = Database result = Ok(()) uri = /instances/978da889-c1eb-4016-b182-d8c4346a3c1d
4222024-02-04T18:31:02.892ZDEBGnexus: deleted metric producer assignment collector_id = 77d82b3e-fc68-4fc7-99ae-c7df6c9287af producer_id = 978da889-c1eb-4016-b182-d8c4346a3c1d
4232024-02-04T18:31:02.924ZINFOnexus: registered oximeter collector client file = nexus/src/app/oximeter.rs:374 id = 978da889-c1eb-4016-b182-d8c4346a3c1d
4242024-02-04T18:31:02.924ZDEBGnexus: client request body = None method = DELETE oximeter-collector = 978da889-c1eb-4016-b182-d8c4346a3c1d uri = http://[fd00:1122:3344:101::d]:12223/producers/978da889-c1eb-4016-b182-d8c4346a3c1d
4252024-02-04T18:31:02.924ZDEBGnexus: client response oximeter-collector = 978da889-c1eb-4016-b182-d8c4346a3c1d result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::d)), port: Some(12223), path: "/producers/978da889-c1eb-4016-b182-d8c4346a3c1d", query: None, fragment: None }, status: 204, headers: {"x-request-id": "74cb029a-089a-40b9-b196-223abbff43be", "date": "Sun, 04 Feb 2024 18:31:02 GMT"} })
4262024-02-04T18:31:02.924ZDEBGnexus: successfully deleted producer from collector address = [fd00:1122:3344:101::d]:12223 collector_id = 77d82b3e-fc68-4fc7-99ae-c7df6c9287af producer_id = 978da889-c1eb-4016-b182-d8c4346a3c1d
4272024-02-04T18:31:02.939ZINFOnexus: vmm is terminated, cleaning up resources file = nexus/src/app/instance.rs:1585 instance_id = 978da889-c1eb-4016-b182-d8c4346a3c1d propolis_id = a99c950c-4d50-4241-8074-58986809f024
4282024-02-04T18:31:02.939ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:51916 req_id = 063dfd49-22df-4e7e-914d-ec2d8424b16d roles = RoleSet { roles: {} } uri = /instances/978da889-c1eb-4016-b182-d8c4346a3c1d
4292024-02-04T18:31:02.939ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:51916 req_id = 063dfd49-22df-4e7e-914d-ec2d8424b16d resource = Database result = Ok(()) uri = /instances/978da889-c1eb-4016-b182-d8c4346a3c1d
4302024-02-04T18:31:02.945ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:51916 req_id = 063dfd49-22df-4e7e-914d-ec2d8424b16d roles = RoleSet { roles: {} } uri = /instances/978da889-c1eb-4016-b182-d8c4346a3c1d
4312024-02-04T18:31:02.945ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:51916 req_id = 063dfd49-22df-4e7e-914d-ec2d8424b16d resource = Database result = Ok(()) uri = /instances/978da889-c1eb-4016-b182-d8c4346a3c1d
4322024-02-04T18:31:02.949ZINFOnexus: instance and vmm updated by sled agent file = nexus/src/app/instance.rs:1608 instance_id = 978da889-c1eb-4016-b182-d8c4346a3c1d instance_updated = true propolis_id = a99c950c-4d50-4241-8074-58986809f024 vmm_updated = true
4332024-02-04T18:31:02.949ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 167099 local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:51916 req_id = 063dfd49-22df-4e7e-914d-ec2d8424b16d response_code = 204 uri = /instances/978da889-c1eb-4016-b182-d8c4346a3c1d
4342024-02-04T18:31:03.232ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:44921 req_id = 963f8348-19bd-45d4-8dce-edf3711fe63b roles = RoleSet { roles: {} } uri = /nat/ipv4/changeset/16?limit=100
4352024-02-04T18:31:03.232ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:44921 req_id = 963f8348-19bd-45d4-8dce-edf3711fe63b resource = Database result = Ok(()) uri = /nat/ipv4/changeset/16?limit=100
4362024-02-04T18:31:03.234ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 1817 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:44921 req_id = 963f8348-19bd-45d4-8dce-edf3711fe63b response_code = 200 uri = /nat/ipv4/changeset/16?limit=100
4372024-02-04T18:31:04.018ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 1599 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:47021 req_id = cf930df9-1ca5-4e6b-83a7-46e234ab3a4a response_code = 200 uri = /metrics/collect/092d8721-ad83-4ea8-800d-5dfcaa02a610
4382024-02-04T18:31:05.764ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 45 reason = Timeout
4392024-02-04T18:31:05.764ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4402024-02-04T18:31:05.764ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
4412024-02-04T18:31:05.766ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4422024-02-04T18:31:05.769ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
4432024-02-04T18:31:05.769ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4442024-02-04T18:31:05.769ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
4452024-02-04T18:31:05.771ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 6.612964ms iteration = 45
4462024-02-04T18:31:14.019ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 1580 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:47021 req_id = dbcf3eec-8ec1-4cfa-9328-e9002008f893 response_code = 200 uri = /metrics/collect/092d8721-ad83-4ea8-800d-5dfcaa02a610
4472024-02-04T18:31:15.764ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 46 reason = Timeout
4482024-02-04T18:31:15.764ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4492024-02-04T18:31:15.764ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
4502024-02-04T18:31:15.766ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4512024-02-04T18:31:15.769ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
4522024-02-04T18:31:15.769ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4532024-02-04T18:31:15.769ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
4542024-02-04T18:31:15.771ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 6.704605ms iteration = 46
4552024-02-04T18:31:24.019ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 1201 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:47021 req_id = 03c00719-b92a-4a90-a95a-04aa268edb78 response_code = 200 uri = /metrics/collect/092d8721-ad83-4ea8-800d-5dfcaa02a610
4562024-02-04T18:31:25.753ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 17 reason = Timeout
4572024-02-04T18:31:25.753ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
4582024-02-04T18:31:25.754ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
4592024-02-04T18:31:25.755ZDEBGnexus (DpdClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
4602024-02-04T18:31:25.755ZDEBGnexus (DpdClient): client response result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "2e5bd9f8-f45a-4cb4-bbaa-6a9852b6e278", "content-length": "2", "date": "Sun, 04 Feb 2024 18:31:25 GMT"} })
4612024-02-04T18:31:25.755ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
4622024-02-04T18:31:25.756ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
4632024-02-04T18:31:25.757ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 3.807908ms iteration = 17
4642024-02-04T18:31:25.764ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 47 reason = Timeout
4652024-02-04T18:31:25.764ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4662024-02-04T18:31:25.764ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 17 reason = Timeout
4672024-02-04T18:31:25.764ZWARNnexus (BackgroundTasks): phantom disk task started background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:46
4682024-02-04T18:31:25.764ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 17 reason = Timeout
4692024-02-04T18:31:25.764ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 17 reason = Timeout
4702024-02-04T18:31:25.764ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
4712024-02-04T18:31:25.764ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
4722024-02-04T18:31:25.765ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
4732024-02-04T18:31:25.765ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
4742024-02-04T18:31:25.765ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
4752024-02-04T18:31:25.766ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4762024-02-04T18:31:25.766ZDEBGnexus: client request MgAdminClient = [fd00:1122:3344:101::2]:4676 body = None method = GET uri = http://[fd00:1122:3344:101::2]:4676/bfd/peers
4772024-02-04T18:31:25.766ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4782024-02-04T18:31:25.767ZDEBGnexus: client response MgAdminClient = [fd00:1122:3344:101::2]:4676 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(4676), path: "/bfd/peers", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "a9f73025-3915-4e5e-88e0-91f358a10d17", "content-length": "2", "date": "Sun, 04 Feb 2024 18:31:25 GMT"} })
4792024-02-04T18:31:25.767ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 2.33889ms iteration = 17
4802024-02-04T18:31:25.767ZWARNnexus (BackgroundTasks): phantom disk task done background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:92
4812024-02-04T18:31:25.767ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.214141ms iteration = 17
4822024-02-04T18:31:25.769ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
4832024-02-04T18:31:25.769ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
4842024-02-04T18:31:25.769ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
4852024-02-04T18:31:25.770ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
4862024-02-04T18:31:25.770ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4872024-02-04T18:31:25.770ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
4882024-02-04T18:31:25.771ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
4892024-02-04T18:31:25.771ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 6.829316ms iteration = 47
4902024-02-04T18:31:25.771ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
4912024-02-04T18:31:25.781ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
4922024-02-04T18:31:25.781ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
4932024-02-04T18:31:25.782ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
4942024-02-04T18:31:25.783ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
4952024-02-04T18:31:25.783ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4962024-02-04T18:31:25.786ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 339b514e-8533-421a-a5ed-83b7bd206824, lookup_type: ById(339b514e-8533-421a-a5ed-83b7bd206824) } result = Ok(())
4972024-02-04T18:31:25.786ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
4982024-02-04T18:31:25.786ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
4992024-02-04T18:31:25.787ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 22.523555ms iteration = 17
5002024-02-04T18:31:33.267ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:44921 req_id = 26f16e00-8ff6-4a64-8491-b375c77dc044 roles = RoleSet { roles: {} } uri = /nat/ipv4/changeset/16?limit=100
5012024-02-04T18:31:33.268ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:44921 req_id = 26f16e00-8ff6-4a64-8491-b375c77dc044 resource = Database result = Ok(()) uri = /nat/ipv4/changeset/16?limit=100
5022024-02-04T18:31:33.270ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 2512 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:44921 req_id = 26f16e00-8ff6-4a64-8491-b375c77dc044 response_code = 200 uri = /nat/ipv4/changeset/16?limit=100
5032024-02-04T18:31:34.020ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 1933 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:47021 req_id = c160a5da-997b-4bbc-9c02-fcf584f47ab5 response_code = 200 uri = /metrics/collect/092d8721-ad83-4ea8-800d-5dfcaa02a610
5042024-02-04T18:31:35.764ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 48 reason = Timeout
5052024-02-04T18:31:35.764ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5062024-02-04T18:31:35.764ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5072024-02-04T18:31:35.766ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5082024-02-04T18:31:35.769ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
5092024-02-04T18:31:35.769ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5102024-02-04T18:31:35.769ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5112024-02-04T18:31:35.771ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 7.306303ms iteration = 48
5122024-02-04T18:31:44.019ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 1659 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:47021 req_id = aeec4a9d-8add-4812-97bd-1c811a96e822 response_code = 200 uri = /metrics/collect/092d8721-ad83-4ea8-800d-5dfcaa02a610
5132024-02-04T18:31:45.765ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 49 reason = Timeout
5142024-02-04T18:31:45.765ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5152024-02-04T18:31:45.765ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5162024-02-04T18:31:45.767ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5172024-02-04T18:31:45.769ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
5182024-02-04T18:31:45.769ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5192024-02-04T18:31:45.769ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5202024-02-04T18:31:45.771ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 6.687234ms iteration = 49
5212024-02-04T18:31:54.019ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 1668 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:47021 req_id = e46b6d4d-7857-4137-9a5c-c9e1d895ee35 response_code = 200 uri = /metrics/collect/092d8721-ad83-4ea8-800d-5dfcaa02a610
5222024-02-04T18:31:55.753ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 11 reason = Timeout
5232024-02-04T18:31:55.753ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 11 reason = Timeout
5242024-02-04T18:31:55.753ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
5252024-02-04T18:31:55.753ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 11 reason = Timeout
5262024-02-04T18:31:55.753ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_internal dns_group = internal roles = RoleSet { roles: {} }
5272024-02-04T18:31:55.753ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 11 reason = Timeout
5282024-02-04T18:31:55.753ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
5292024-02-04T18:31:55.753ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
5302024-02-04T18:31:55.753ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
5312024-02-04T18:31:55.753ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
5322024-02-04T18:31:55.753ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 11 reason = Timeout
5332024-02-04T18:31:55.753ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_external dns_group = external roles = RoleSet { roles: {} }
5342024-02-04T18:31:55.753ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
5352024-02-04T18:31:55.753ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_external dns_group = external resource = Database result = Ok(())
5362024-02-04T18:31:55.753ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_internal dns_group = internal resource = Database result = Ok(())
5372024-02-04T18:31:55.753ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
5382024-02-04T18:31:55.753ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
5392024-02-04T18:31:55.754ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
5402024-02-04T18:31:55.754ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 18 reason = Timeout
5412024-02-04T18:31:55.754ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
5422024-02-04T18:31:55.754ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
5432024-02-04T18:31:55.754ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
5442024-02-04T18:31:55.754ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5452024-02-04T18:31:55.755ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
5462024-02-04T18:31:55.755ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5472024-02-04T18:31:55.755ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5482024-02-04T18:31:55.755ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
5492024-02-04T18:31:55.755ZDEBGnexus (DpdClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
5502024-02-04T18:31:55.756ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5512024-02-04T18:31:55.756ZDEBGnexus (DpdClient): client response result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "8af47898-82f2-4f4b-b12c-698de92b9f1c", "content-length": "2", "date": "Sun, 04 Feb 2024 18:31:55 GMT"} })
5522024-02-04T18:31:55.756ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
5532024-02-04T18:31:55.756ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5542024-02-04T18:31:55.756ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
5552024-02-04T18:31:55.757ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
5562024-02-04T18:31:55.757ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
5572024-02-04T18:31:55.757ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
5582024-02-04T18:31:55.757ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 3.770538ms iteration = 18
5592024-02-04T18:31:55.758ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_internal dns_group = internal resource = Fleet result = Ok(())
5602024-02-04T18:31:55.758ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_internal dns_group = internal roles = RoleSet { roles: {} }
5612024-02-04T18:31:55.758ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_external dns_group = external resource = Fleet result = Ok(())
5622024-02-04T18:31:55.758ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_external dns_group = external roles = RoleSet { roles: {} }
5632024-02-04T18:31:55.758ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_internal dns_group = internal resource = Database result = Ok(())
5642024-02-04T18:31:55.758ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_external dns_group = external resource = Database result = Ok(())
5652024-02-04T18:31:55.759ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
5662024-02-04T18:31:55.759ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
5672024-02-04T18:31:55.759ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } dns_group = external
5682024-02-04T18:31:55.759ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 6.708265ms iteration = 11
5692024-02-04T18:31:55.760ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } dns_group = internal
5702024-02-04T18:31:55.760ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 7.010079ms iteration = 11
5712024-02-04T18:31:55.760ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
5722024-02-04T18:31:55.760ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5732024-02-04T18:31:55.761ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
5742024-02-04T18:31:55.761ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
5752024-02-04T18:31:55.761ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
5762024-02-04T18:31:55.761ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
5772024-02-04T18:31:55.761ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
5782024-02-04T18:31:55.761ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
5792024-02-04T18:31:55.762ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
5802024-02-04T18:31:55.762ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
5812024-02-04T18:31:55.762ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
5822024-02-04T18:31:55.762ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
5832024-02-04T18:31:55.762ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5842024-02-04T18:31:55.762ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
5852024-02-04T18:31:55.763ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5862024-02-04T18:31:55.764ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
5872024-02-04T18:31:55.764ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
5882024-02-04T18:31:55.764ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
5892024-02-04T18:31:55.765ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 50 reason = Timeout
5902024-02-04T18:31:55.765ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 13 reason = Timeout
5912024-02-04T18:31:55.765ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 10 reason = Timeout
5922024-02-04T18:31:55.765ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/blueprint_execution.rs:46 reason = no blueprint
5932024-02-04T18:31:55.765ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5942024-02-04T18:31:55.765ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 5.38\u{b5}s iteration = 10
5952024-02-04T18:31:55.765ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 13 reason = Timeout
5962024-02-04T18:31:55.765ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 18 reason = Timeout
5972024-02-04T18:31:55.765ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
5982024-02-04T18:31:55.765ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 18 reason = Timeout
5992024-02-04T18:31:55.765ZWARNnexus (BackgroundTasks): phantom disk task started background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:46
6002024-02-04T18:31:55.765ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 18 reason = Timeout
6012024-02-04T18:31:55.765ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
6022024-02-04T18:31:55.765ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
6032024-02-04T18:31:55.765ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
6042024-02-04T18:31:55.765ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
6052024-02-04T18:31:55.765ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6062024-02-04T18:31:55.765ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
6072024-02-04T18:31:55.765ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
6082024-02-04T18:31:55.765ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
6092024-02-04T18:31:55.766ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
6102024-02-04T18:31:55.766ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6112024-02-04T18:31:55.767ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
6122024-02-04T18:31:55.767ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
6132024-02-04T18:31:55.767ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6142024-02-04T18:31:55.767ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
6152024-02-04T18:31:55.767ZDEBGnexus: client request MgAdminClient = [fd00:1122:3344:101::2]:4676 body = None method = GET uri = http://[fd00:1122:3344:101::2]:4676/bfd/peers
6162024-02-04T18:31:55.767ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6172024-02-04T18:31:55.767ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
6182024-02-04T18:31:55.767ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 569ffb23-fbba-49ec-b235-2603da0b3eed dns_zone_name = oxide.test found_so_far = 0 version = 2
6192024-02-04T18:31:55.767ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
6202024-02-04T18:31:55.767ZWARNnexus (BackgroundTasks): phantom disk task done background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:92
6212024-02-04T18:31:55.767ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 2.728645ms iteration = 18
6222024-02-04T18:31:55.768ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
6232024-02-04T18:31:55.768ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
6242024-02-04T18:31:55.768ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 64c3c424-e322-4f8a-b3f5-aea263693dd8 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
6252024-02-04T18:31:55.768ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
6262024-02-04T18:31:55.769ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
6272024-02-04T18:31:55.769ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6282024-02-04T18:31:55.769ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
6292024-02-04T18:31:55.769ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
6302024-02-04T18:31:55.769ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
6312024-02-04T18:31:55.770ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6322024-02-04T18:31:55.772ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
6332024-02-04T18:31:55.772ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
6342024-02-04T18:31:55.772ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
6352024-02-04T18:31:55.772ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6362024-02-04T18:31:55.772ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
6372024-02-04T18:31:55.773ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6382024-02-04T18:31:55.773ZDEBGnexus: client response MgAdminClient = [fd00:1122:3344:101::2]:4676 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(4676), path: "/bfd/peers", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "5eee9731-738b-4630-bac7-53a4c159a2d6", "content-length": "2", "date": "Sun, 04 Feb 2024 18:31:55 GMT"} })
6392024-02-04T18:31:55.773ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 8.028672ms iteration = 18
6402024-02-04T18:31:55.774ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
6412024-02-04T18:31:55.774ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
6422024-02-04T18:31:55.774ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
6432024-02-04T18:31:55.774ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
6442024-02-04T18:31:55.774ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
6452024-02-04T18:31:55.774ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2024-02-04T18:23:55.840039Z, time_modified: 2024-02-04T18:23:55.840039Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 7eaca1c5-9568-4aa5-a1b2-cb905947fbfb, db_silo: Silo { identity: SiloIdentity { id: 7eaca1c5-9568-4aa5-a1b2-cb905947fbfb, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-02-04T18:24:12.352343Z, time_modified: 2024-02-04T18:24:12.352343Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "209b27934f4c6206a8d147070bebbcc41c8953fdb7a37a8a7847a104d18adc4c" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 7eaca1c5-9568-4aa5-a1b2-cb905947fbfb, db_silo: Silo { identity: SiloIdentity { id: 7eaca1c5-9568-4aa5-a1b2-cb905947fbfb, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-02-04T18:24:12.352343Z, time_modified: 2024-02-04T18:24:12.352343Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "209b27934f4c6206a8d147070bebbcc41c8953fdb7a37a8a7847a104d18adc4c" }] }) }
6462024-02-04T18:31:55.774ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 21.609163ms iteration = 11
6472024-02-04T18:31:55.774ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
6482024-02-04T18:31:55.774ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
6492024-02-04T18:31:55.774ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
6502024-02-04T18:31:55.775ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 9.964306ms iteration = 50
6512024-02-04T18:31:55.775ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 569ffb23-fbba-49ec-b235-2603da0b3eed dns_zone_name = oxide.test found_so_far = 1 version = 2
6522024-02-04T18:31:55.775ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
6532024-02-04T18:31:55.775ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2024-02-04 18:24:12.376697 UTC dns_group = external generation = 2
6542024-02-04T18:31:55.775ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 22.889499ms iteration = 11
6552024-02-04T18:31:55.776ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 64c3c424-e322-4f8a-b3f5-aea263693dd8 dns_zone_name = control-plane.oxide.internal found_so_far = 51 version = 1
6562024-02-04T18:31:55.776ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
6572024-02-04T18:31:55.776ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2024-02-04 18:24:12.000370 UTC dns_group = internal generation = 1
6582024-02-04T18:31:55.776ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 23.120392ms iteration = 11
6592024-02-04T18:31:55.788ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
6602024-02-04T18:31:55.789ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
6612024-02-04T18:31:55.790ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
6622024-02-04T18:31:55.790ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
6632024-02-04T18:31:55.791ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6642024-02-04T18:31:55.793ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 339b514e-8533-421a-a5ed-83b7bd206824, lookup_type: ById(339b514e-8533-421a-a5ed-83b7bd206824) } result = Ok(())
6652024-02-04T18:31:55.793ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
6662024-02-04T18:31:55.793ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
6672024-02-04T18:31:55.795ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 30.040839ms iteration = 18
6682024-02-04T18:31:55.829ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
6692024-02-04T18:31:55.832ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
6702024-02-04T18:31:55.890ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
6712024-02-04T18:31:55.890ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::8)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "e7ba4bde-0f48-44e2-ba5b-b0442f0f3124", "date": "Sun, 04 Feb 2024 18:31:55 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
6722024-02-04T18:31:55.890ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
6732024-02-04T18:31:55.892ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::9)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "f3836523-5511-4c44-abec-1000591d30c4", "date": "Sun, 04 Feb 2024 18:31:55 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
6742024-02-04T18:31:55.892ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
6752024-02-04T18:31:55.893ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
6762024-02-04T18:31:55.893ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 128.600935ms iteration = 13
6772024-02-04T18:31:55.894ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
6782024-02-04T18:31:55.925ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
6792024-02-04T18:31:55.925ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:2::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "1a96ec2b-39aa-4c9d-b097-2584bae1a840", "date": "Sun, 04 Feb 2024 18:31:55 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
6802024-02-04T18:31:55.926ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
6812024-02-04T18:31:55.926ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:1::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "6e50a6bf-336d-4660-af1c-637ffb0d3069", "date": "Sun, 04 Feb 2024 18:31:55 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
6822024-02-04T18:31:55.926ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
6832024-02-04T18:31:55.928ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:3::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "01b6586f-6cdf-4ae6-86a1-bad13da04294", "date": "Sun, 04 Feb 2024 18:31:55 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
6842024-02-04T18:31:55.928ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
6852024-02-04T18:31:55.930ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
6862024-02-04T18:31:55.930ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 164.995264ms iteration = 13
6872024-02-04T18:32:03.300ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:44921 req_id = e2e59387-6c1a-4e68-9682-e7f17c7bb55c roles = RoleSet { roles: {} } uri = /nat/ipv4/changeset/16?limit=100
6882024-02-04T18:32:03.300ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:44921 req_id = e2e59387-6c1a-4e68-9682-e7f17c7bb55c resource = Database result = Ok(()) uri = /nat/ipv4/changeset/16?limit=100
6892024-02-04T18:32:03.302ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 2415 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:44921 req_id = e2e59387-6c1a-4e68-9682-e7f17c7bb55c response_code = 200 uri = /nat/ipv4/changeset/16?limit=100
6902024-02-04T18:32:04.019ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 1221 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:47021 req_id = db3c0ab0-3ad7-4630-9f60-2dde2682e32f response_code = 200 uri = /metrics/collect/092d8721-ad83-4ea8-800d-5dfcaa02a610
6912024-02-04T18:32:05.764ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 51 reason = Timeout
6922024-02-04T18:32:05.764ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6932024-02-04T18:32:05.764ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6942024-02-04T18:32:05.766ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6952024-02-04T18:32:05.768ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
6962024-02-04T18:32:05.768ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6972024-02-04T18:32:05.769ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6982024-02-04T18:32:05.770ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 5.720912ms iteration = 51
6992024-02-04T18:32:14.020ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 1746 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:47021 req_id = cd1ab222-4e52-4541-be49-ebbf0ff8e454 response_code = 200 uri = /metrics/collect/092d8721-ad83-4ea8-800d-5dfcaa02a610
7002024-02-04T18:32:15.764ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 52 reason = Timeout
7012024-02-04T18:32:15.764ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
7022024-02-04T18:32:15.764ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
7032024-02-04T18:32:15.766ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7042024-02-04T18:32:15.769ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
7052024-02-04T18:32:15.769ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
7062024-02-04T18:32:15.769ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
7072024-02-04T18:32:15.770ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 6.358051ms iteration = 52
7082024-02-04T18:32:24.020ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 1678 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:47021 req_id = 4cdc94fc-567e-4355-9fb7-c3960f726537 response_code = 200 uri = /metrics/collect/092d8721-ad83-4ea8-800d-5dfcaa02a610
7092024-02-04T18:32:25.753ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 19 reason = Timeout
7102024-02-04T18:32:25.754ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
7112024-02-04T18:32:25.754ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
7122024-02-04T18:32:25.756ZDEBGnexus (DpdClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
7132024-02-04T18:32:25.756ZDEBGnexus (DpdClient): client response result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "80341065-4cdd-45aa-87df-a4859250aa68", "content-length": "2", "date": "Sun, 04 Feb 2024 18:32:25 GMT"} })
7142024-02-04T18:32:25.756ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
7152024-02-04T18:32:25.756ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
7162024-02-04T18:32:25.757ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 3.478914ms iteration = 19
7172024-02-04T18:32:25.765ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 53 reason = Timeout
7182024-02-04T18:32:25.765ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 19 reason = Timeout
7192024-02-04T18:32:25.765ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
7202024-02-04T18:32:25.765ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
7212024-02-04T18:32:25.765ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 19 reason = Timeout
7222024-02-04T18:32:25.765ZWARNnexus (BackgroundTasks): phantom disk task started background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:46
7232024-02-04T18:32:25.765ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 19 reason = Timeout
7242024-02-04T18:32:25.765ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
7252024-02-04T18:32:25.765ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
7262024-02-04T18:32:25.765ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
7272024-02-04T18:32:25.765ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
7282024-02-04T18:32:25.767ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7292024-02-04T18:32:25.767ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7302024-02-04T18:32:25.767ZDEBGnexus: client request MgAdminClient = [fd00:1122:3344:101::2]:4676 body = None method = GET uri = http://[fd00:1122:3344:101::2]:4676/bfd/peers
7312024-02-04T18:32:25.767ZDEBGnexus: client response MgAdminClient = [fd00:1122:3344:101::2]:4676 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(4676), path: "/bfd/peers", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "1a7eab0e-c09c-4d40-9c38-f33527b97c91", "content-length": "2", "date": "Sun, 04 Feb 2024 18:32:25 GMT"} })
7322024-02-04T18:32:25.767ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 2.353ms iteration = 19
7332024-02-04T18:32:25.768ZWARNnexus (BackgroundTasks): phantom disk task done background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:92
7342024-02-04T18:32:25.768ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 2.723315ms iteration = 19
7352024-02-04T18:32:25.770ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
7362024-02-04T18:32:25.770ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
7372024-02-04T18:32:25.770ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
7382024-02-04T18:32:25.770ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
7392024-02-04T18:32:25.770ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
7402024-02-04T18:32:25.770ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
7412024-02-04T18:32:25.771ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
7422024-02-04T18:32:25.771ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 6.553772ms iteration = 53
7432024-02-04T18:32:25.772ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
7442024-02-04T18:32:25.782ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
7452024-02-04T18:32:25.782ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
7462024-02-04T18:32:25.783ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
7472024-02-04T18:32:25.783ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
7482024-02-04T18:32:25.784ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7492024-02-04T18:32:25.786ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 339b514e-8533-421a-a5ed-83b7bd206824, lookup_type: ById(339b514e-8533-421a-a5ed-83b7bd206824) } result = Ok(())
7502024-02-04T18:32:25.786ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
7512024-02-04T18:32:25.787ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
7522024-02-04T18:32:25.787ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 22.427213ms iteration = 19
7532024-02-04T18:32:33.335ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:44921 req_id = 974931e1-bcf0-49d2-abb3-2702ce6c5821 roles = RoleSet { roles: {} } uri = /nat/ipv4/changeset/16?limit=100
7542024-02-04T18:32:33.335ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:44921 req_id = 974931e1-bcf0-49d2-abb3-2702ce6c5821 resource = Database result = Ok(()) uri = /nat/ipv4/changeset/16?limit=100
7552024-02-04T18:32:33.337ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 2272 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:44921 req_id = 974931e1-bcf0-49d2-abb3-2702ce6c5821 response_code = 200 uri = /nat/ipv4/changeset/16?limit=100
7562024-02-04T18:32:34.020ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 1584 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:47021 req_id = 80a0965c-3fac-4bd5-b536-96273fe8315f response_code = 200 uri = /metrics/collect/092d8721-ad83-4ea8-800d-5dfcaa02a610
7572024-02-04T18:32:35.765ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 54 reason = Timeout
7582024-02-04T18:32:35.765ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
7592024-02-04T18:32:35.766ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
7602024-02-04T18:32:35.768ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7612024-02-04T18:32:35.770ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
7622024-02-04T18:32:35.770ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
7632024-02-04T18:32:35.770ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
7642024-02-04T18:32:35.772ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 6.844386ms iteration = 54
7652024-02-04T18:32:44.019ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 1414 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:47021 req_id = f68ddef5-e4fe-4f0c-9bd6-d7c94360c9c3 response_code = 200 uri = /metrics/collect/092d8721-ad83-4ea8-800d-5dfcaa02a610
7662024-02-04T18:32:45.765ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 55 reason = Timeout
7672024-02-04T18:32:45.765ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
7682024-02-04T18:32:45.765ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
7692024-02-04T18:32:45.767ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7702024-02-04T18:32:45.770ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
7712024-02-04T18:32:45.770ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
7722024-02-04T18:32:45.770ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
7732024-02-04T18:32:45.771ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 6.768885ms iteration = 55
7742024-02-04T18:32:54.020ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 1877 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:47021 req_id = eb549767-b99f-4207-a906-b95c14f8f935 response_code = 200 uri = /metrics/collect/092d8721-ad83-4ea8-800d-5dfcaa02a610
7752024-02-04T18:32:55.753ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 12 reason = Timeout
7762024-02-04T18:32:55.753ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
7772024-02-04T18:32:55.753ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 12 reason = Timeout
7782024-02-04T18:32:55.753ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
7792024-02-04T18:32:55.753ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 12 reason = Timeout
7802024-02-04T18:32:55.753ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_internal dns_group = internal roles = RoleSet { roles: {} }
7812024-02-04T18:32:55.753ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
7822024-02-04T18:32:55.753ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 12 reason = Timeout
7832024-02-04T18:32:55.753ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
7842024-02-04T18:32:55.753ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 12 reason = Timeout
7852024-02-04T18:32:55.753ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
7862024-02-04T18:32:55.753ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_external dns_group = external roles = RoleSet { roles: {} }
7872024-02-04T18:32:55.753ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_internal dns_group = internal resource = Database result = Ok(())
7882024-02-04T18:32:55.753ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
7892024-02-04T18:32:55.753ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
7902024-02-04T18:32:55.753ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_external dns_group = external resource = Database result = Ok(())
7912024-02-04T18:32:55.753ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
7922024-02-04T18:32:55.754ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 20 reason = Timeout
7932024-02-04T18:32:55.754ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
7942024-02-04T18:32:55.754ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
7952024-02-04T18:32:55.755ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7962024-02-04T18:32:55.755ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
7972024-02-04T18:32:55.755ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
7982024-02-04T18:32:55.755ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
7992024-02-04T18:32:55.755ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8002024-02-04T18:32:55.755ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8012024-02-04T18:32:55.756ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
8022024-02-04T18:32:55.756ZDEBGnexus (DpdClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
8032024-02-04T18:32:55.756ZDEBGnexus (DpdClient): client response result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "3fe2151f-3bfa-4a7e-87b8-189e0bad12b6", "content-length": "2", "date": "Sun, 04 Feb 2024 18:32:55 GMT"} })
8042024-02-04T18:32:55.756ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
8052024-02-04T18:32:55.756ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8062024-02-04T18:32:55.757ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8072024-02-04T18:32:55.757ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
8082024-02-04T18:32:55.758ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_internal dns_group = internal resource = Fleet result = Ok(())
8092024-02-04T18:32:55.758ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_internal dns_group = internal roles = RoleSet { roles: {} }
8102024-02-04T18:32:55.758ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 3.912739ms iteration = 20
8112024-02-04T18:32:55.758ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_internal dns_group = internal resource = Database result = Ok(())
8122024-02-04T18:32:55.758ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
8132024-02-04T18:32:55.758ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
8142024-02-04T18:32:55.758ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
8152024-02-04T18:32:55.759ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_external dns_group = external resource = Fleet result = Ok(())
8162024-02-04T18:32:55.759ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_external dns_group = external roles = RoleSet { roles: {} }
8172024-02-04T18:32:55.759ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_external dns_group = external resource = Database result = Ok(())
8182024-02-04T18:32:55.759ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } dns_group = internal
8192024-02-04T18:32:55.759ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 6.409391ms iteration = 12
8202024-02-04T18:32:55.760ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } dns_group = external
8212024-02-04T18:32:55.760ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 7.002858ms iteration = 12
8222024-02-04T18:32:55.760ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
8232024-02-04T18:32:55.760ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
8242024-02-04T18:32:55.761ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
8252024-02-04T18:32:55.761ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
8262024-02-04T18:32:55.761ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
8272024-02-04T18:32:55.761ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
8282024-02-04T18:32:55.761ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
8292024-02-04T18:32:55.761ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
8302024-02-04T18:32:55.762ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8312024-02-04T18:32:55.762ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
8322024-02-04T18:32:55.762ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
8332024-02-04T18:32:55.762ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
8342024-02-04T18:32:55.762ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
8352024-02-04T18:32:55.762ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
8362024-02-04T18:32:55.762ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
8372024-02-04T18:32:55.763ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8382024-02-04T18:32:55.763ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8392024-02-04T18:32:55.765ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
8402024-02-04T18:32:55.765ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
8412024-02-04T18:32:55.765ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
8422024-02-04T18:32:55.765ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 56 reason = Timeout
8432024-02-04T18:32:55.765ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 14 reason = Timeout
8442024-02-04T18:32:55.765ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
8452024-02-04T18:32:55.765ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 20 reason = Timeout
8462024-02-04T18:32:55.765ZWARNnexus (BackgroundTasks): phantom disk task started background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:46
8472024-02-04T18:32:55.765ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 20 reason = Timeout
8482024-02-04T18:32:55.765ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 20 reason = Timeout
8492024-02-04T18:32:55.765ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
8502024-02-04T18:32:55.765ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
8512024-02-04T18:32:55.765ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 11 reason = Timeout
8522024-02-04T18:32:55.765ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/blueprint_execution.rs:46 reason = no blueprint
8532024-02-04T18:32:55.765ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 6.9\u{b5}s iteration = 11
8542024-02-04T18:32:55.765ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 14 reason = Timeout
8552024-02-04T18:32:55.765ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
8562024-02-04T18:32:55.765ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
8572024-02-04T18:32:55.765ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
8582024-02-04T18:32:55.766ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
8592024-02-04T18:32:55.767ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8602024-02-04T18:32:55.767ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
8612024-02-04T18:32:55.767ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
8622024-02-04T18:32:55.767ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
8632024-02-04T18:32:55.767ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
8642024-02-04T18:32:55.767ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8652024-02-04T18:32:55.767ZDEBGnexus: client request MgAdminClient = [fd00:1122:3344:101::2]:4676 body = None method = GET uri = http://[fd00:1122:3344:101::2]:4676/bfd/peers
8662024-02-04T18:32:55.767ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
8672024-02-04T18:32:55.767ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
8682024-02-04T18:32:55.767ZDEBGnexus: client response MgAdminClient = [fd00:1122:3344:101::2]:4676 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(4676), path: "/bfd/peers", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "45767126-2a22-4238-835c-26b18b764aeb", "content-length": "2", "date": "Sun, 04 Feb 2024 18:32:55 GMT"} })
8692024-02-04T18:32:55.767ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 2.166697ms iteration = 20
8702024-02-04T18:32:55.767ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
8712024-02-04T18:32:55.768ZWARNnexus (BackgroundTasks): phantom disk task done background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:92
8722024-02-04T18:32:55.768ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 2.593022ms iteration = 20
8732024-02-04T18:32:55.768ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8742024-02-04T18:32:55.769ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
8752024-02-04T18:32:55.769ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 64c3c424-e322-4f8a-b3f5-aea263693dd8 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
8762024-02-04T18:32:55.769ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
8772024-02-04T18:32:55.769ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
8782024-02-04T18:32:55.769ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
8792024-02-04T18:32:55.769ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 569ffb23-fbba-49ec-b235-2603da0b3eed dns_zone_name = oxide.test found_so_far = 0 version = 2
8802024-02-04T18:32:55.769ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
8812024-02-04T18:32:55.769ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
8822024-02-04T18:32:55.770ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8832024-02-04T18:32:55.770ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8842024-02-04T18:32:55.770ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
8852024-02-04T18:32:55.771ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
8862024-02-04T18:32:55.771ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
8872024-02-04T18:32:55.771ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
8882024-02-04T18:32:55.771ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
8892024-02-04T18:32:55.771ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
8902024-02-04T18:32:55.771ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
8912024-02-04T18:32:55.771ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
8922024-02-04T18:32:55.771ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
8932024-02-04T18:32:55.773ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 7.715658ms iteration = 56
8942024-02-04T18:32:55.773ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
8952024-02-04T18:32:55.773ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
8962024-02-04T18:32:55.774ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
8972024-02-04T18:32:55.774ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
8982024-02-04T18:32:55.774ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
8992024-02-04T18:32:55.774ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
9002024-02-04T18:32:55.774ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
9012024-02-04T18:32:55.774ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
9022024-02-04T18:32:55.775ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 569ffb23-fbba-49ec-b235-2603da0b3eed dns_zone_name = oxide.test found_so_far = 1 version = 2
9032024-02-04T18:32:55.775ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
9042024-02-04T18:32:55.775ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2024-02-04 18:24:12.376697 UTC dns_group = external generation = 2
9052024-02-04T18:32:55.775ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 22.152739ms iteration = 12
9062024-02-04T18:32:55.776ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 64c3c424-e322-4f8a-b3f5-aea263693dd8 dns_zone_name = control-plane.oxide.internal found_so_far = 51 version = 1
9072024-02-04T18:32:55.776ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
9082024-02-04T18:32:55.776ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2024-02-04 18:24:12.000370 UTC dns_group = internal generation = 1
9092024-02-04T18:32:55.776ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 23.191663ms iteration = 12
9102024-02-04T18:32:55.780ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2024-02-04T18:23:55.840039Z, time_modified: 2024-02-04T18:23:55.840039Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 7eaca1c5-9568-4aa5-a1b2-cb905947fbfb, db_silo: Silo { identity: SiloIdentity { id: 7eaca1c5-9568-4aa5-a1b2-cb905947fbfb, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-02-04T18:24:12.352343Z, time_modified: 2024-02-04T18:24:12.352343Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "209b27934f4c6206a8d147070bebbcc41c8953fdb7a37a8a7847a104d18adc4c" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 7eaca1c5-9568-4aa5-a1b2-cb905947fbfb, db_silo: Silo { identity: SiloIdentity { id: 7eaca1c5-9568-4aa5-a1b2-cb905947fbfb, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-02-04T18:24:12.352343Z, time_modified: 2024-02-04T18:24:12.352343Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "209b27934f4c6206a8d147070bebbcc41c8953fdb7a37a8a7847a104d18adc4c" }] }) }
9112024-02-04T18:32:55.780ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 27.340496ms iteration = 12
9122024-02-04T18:32:55.789ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
9132024-02-04T18:32:55.789ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
9142024-02-04T18:32:55.790ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
9152024-02-04T18:32:55.790ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
9162024-02-04T18:32:55.792ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9172024-02-04T18:32:55.794ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 339b514e-8533-421a-a5ed-83b7bd206824, lookup_type: ById(339b514e-8533-421a-a5ed-83b7bd206824) } result = Ok(())
9182024-02-04T18:32:55.794ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
9192024-02-04T18:32:55.794ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
9202024-02-04T18:32:55.795ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 30.298802ms iteration = 20
9212024-02-04T18:32:55.832ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
9222024-02-04T18:32:55.836ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
9232024-02-04T18:32:55.897ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
9242024-02-04T18:32:55.900ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
9252024-02-04T18:32:55.900ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::8)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "50a844b5-aeca-4d38-8e23-e1e028058f9d", "date": "Sun, 04 Feb 2024 18:32:55 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
9262024-02-04T18:32:55.900ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
9272024-02-04T18:32:55.901ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::9)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "d01b48df-5d15-4a2a-9f91-6b45ef2c29e5", "date": "Sun, 04 Feb 2024 18:32:55 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
9282024-02-04T18:32:55.901ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
9292024-02-04T18:32:55.903ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
9302024-02-04T18:32:55.903ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 137.853642ms iteration = 14
9312024-02-04T18:32:55.929ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
9322024-02-04T18:32:55.929ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:2::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "571e83c8-06f5-480e-8b8d-5792e20605ac", "date": "Sun, 04 Feb 2024 18:32:55 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
9332024-02-04T18:32:55.929ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
9342024-02-04T18:32:55.931ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:1::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "8c497bca-2b1b-44d4-8f51-3860166c9e12", "date": "Sun, 04 Feb 2024 18:32:55 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
9352024-02-04T18:32:55.931ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
9362024-02-04T18:32:55.933ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:3::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "e7921e6b-c029-4600-aa0d-ccfa5406d62e", "date": "Sun, 04 Feb 2024 18:32:55 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
9372024-02-04T18:32:55.933ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
9382024-02-04T18:32:55.935ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
9392024-02-04T18:32:55.935ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 169.714134ms iteration = 14