Max level shown:
12023-12-12T05:45:12.279ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 758 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:36542 req_id = 4e786d37-6a6c-40e3-97cc-09dd7906cbd3 response_code = 200 uri = /metrics/collect/47e44ab6-ee0b-4407-95b8-c3af199399c8
22023-12-12T05:45:22.279ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 1361 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:36542 req_id = 2c14257d-c833-4f8a-997f-86d6a8816073 response_code = 200 uri = /metrics/collect/47e44ab6-ee0b-4407-95b8-c3af199399c8
32023-12-12T05:45:32.036ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 7 reason = Timeout
42023-12-12T05:45:32.037ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
52023-12-12T05:45:32.037ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 7 reason = Timeout
62023-12-12T05:45:32.037ZWARNnexus (BackgroundTasks): phantom disk task started background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:50
72023-12-12T05:45:32.037ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
82023-12-12T05:45:32.039ZDEBGnexus (DpdClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
92023-12-12T05:45:32.039ZDEBGnexus (DpdClient): client response result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "165a4fa3-08fc-4c95-bd9b-a5158934ad9b", "content-length": "1", "date": "Tue, 12 Dec 2023 05:45:31 GMT"} })
102023-12-12T05:45:32.039ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
112023-12-12T05:45:32.039ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
122023-12-12T05:45:32.039ZWARNnexus (BackgroundTasks): phantom disk task done background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:96
132023-12-12T05:45:32.039ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 2.795423ms iteration = 7
142023-12-12T05:45:32.040ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 3.800274ms iteration = 7
152023-12-12T05:45:32.279ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 509 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:36542 req_id = eb78b57c-18bd-4375-a349-d24b5006d746 response_code = 200 uri = /metrics/collect/47e44ab6-ee0b-4407-95b8-c3af199399c8
162023-12-12T05:45:42.278ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 774 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:36542 req_id = f206b98f-de6e-4e05-bbcb-7b38f503d2cf response_code = 200 uri = /metrics/collect/47e44ab6-ee0b-4407-95b8-c3af199399c8
172023-12-12T05:45:52.280ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 417 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:36542 req_id = edefedc1-c50c-4c45-9393-ee610c0ab9fd response_code = 200 uri = /metrics/collect/47e44ab6-ee0b-4407-95b8-c3af199399c8
182023-12-12T05:46:02.037ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 8 reason = Timeout
192023-12-12T05:46:02.037ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 6 reason = Timeout
202023-12-12T05:46:02.037ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
212023-12-12T05:46:02.037ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 6 reason = Timeout
222023-12-12T05:46:02.037ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
232023-12-12T05:46:02.037ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 6 reason = Timeout
242023-12-12T05:46:02.037ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 6 reason = Timeout
252023-12-12T05:46:02.037ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_external dns_group = external roles = RoleSet { roles: {} }
262023-12-12T05:46:02.037ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_internal dns_group = internal roles = RoleSet { roles: {} }
272023-12-12T05:46:02.037ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
282023-12-12T05:46:02.037ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 8 reason = Timeout
292023-12-12T05:46:02.037ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 6 reason = Timeout
302023-12-12T05:46:02.037ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
312023-12-12T05:46:02.037ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
322023-12-12T05:46:02.037ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
332023-12-12T05:46:02.037ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_internal dns_group = internal resource = Database result = Ok(())
342023-12-12T05:46:02.037ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
352023-12-12T05:46:02.037ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
362023-12-12T05:46:02.037ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_external dns_group = external resource = Database result = Ok(())
372023-12-12T05:46:02.038ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
382023-12-12T05:46:02.038ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
392023-12-12T05:46:02.038ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
402023-12-12T05:46:02.039ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
412023-12-12T05:46:02.039ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 8 reason = Timeout
422023-12-12T05:46:02.039ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
432023-12-12T05:46:02.039ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 8 reason = Timeout
442023-12-12T05:46:02.039ZWARNnexus (BackgroundTasks): phantom disk task started background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:50
452023-12-12T05:46:02.039ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
462023-12-12T05:46:02.039ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
472023-12-12T05:46:02.039ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
482023-12-12T05:46:02.040ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
492023-12-12T05:46:02.040ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
502023-12-12T05:46:02.040ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
512023-12-12T05:46:02.041ZDEBGnexus (DpdClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
522023-12-12T05:46:02.041ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
532023-12-12T05:46:02.041ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
542023-12-12T05:46:02.042ZDEBGnexus (DpdClient): client response result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "e5d2d63b-ea0e-49ef-b9c0-2a281a8cffba", "content-length": "1", "date": "Tue, 12 Dec 2023 05:46:01 GMT"} })
552023-12-12T05:46:02.042ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
562023-12-12T05:46:02.042ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
572023-12-12T05:46:02.042ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
582023-12-12T05:46:02.042ZWARNnexus (BackgroundTasks): phantom disk task done background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:96
592023-12-12T05:46:02.042ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.38113ms iteration = 8
602023-12-12T05:46:02.042ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_external dns_group = external resource = Fleet result = Ok(())
612023-12-12T05:46:02.042ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_external dns_group = external roles = RoleSet { roles: {} }
622023-12-12T05:46:02.042ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_external dns_group = external resource = Database result = Ok(())
632023-12-12T05:46:02.043ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_internal dns_group = internal resource = Fleet result = Ok(())
642023-12-12T05:46:02.043ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_internal dns_group = internal roles = RoleSet { roles: {} }
652023-12-12T05:46:02.043ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_internal dns_group = internal resource = Database result = Ok(())
662023-12-12T05:46:02.043ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 4.804266ms iteration = 8
672023-12-12T05:46:02.044ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
682023-12-12T05:46:02.044ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
692023-12-12T05:46:02.044ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } dns_group = external
702023-12-12T05:46:02.044ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 7.576449ms iteration = 6
712023-12-12T05:46:02.044ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
722023-12-12T05:46:02.045ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
732023-12-12T05:46:02.045ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
742023-12-12T05:46:02.045ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
752023-12-12T05:46:02.045ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
762023-12-12T05:46:02.045ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
772023-12-12T05:46:02.045ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
782023-12-12T05:46:02.045ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
792023-12-12T05:46:02.045ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } dns_group = internal
802023-12-12T05:46:02.046ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 8.921755ms iteration = 6
812023-12-12T05:46:02.046ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
822023-12-12T05:46:02.048ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
832023-12-12T05:46:02.048ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
842023-12-12T05:46:02.048ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
852023-12-12T05:46:02.049ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
862023-12-12T05:46:02.049ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
872023-12-12T05:46:02.049ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
882023-12-12T05:46:02.050ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
892023-12-12T05:46:02.050ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
902023-12-12T05:46:02.051ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
912023-12-12T05:46:02.051ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 54229ede-970c-45c1-9159-c834119f9f05 dns_zone_name = oxide.test found_so_far = 0 version = 2
922023-12-12T05:46:02.051ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
932023-12-12T05:46:02.051ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
942023-12-12T05:46:02.051ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
952023-12-12T05:46:02.052ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
962023-12-12T05:46:02.053ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
972023-12-12T05:46:02.053ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
982023-12-12T05:46:02.054ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
992023-12-12T05:46:02.055ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
1002023-12-12T05:46:02.055ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
1012023-12-12T05:46:02.055ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
1022023-12-12T05:46:02.056ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 54229ede-970c-45c1-9159-c834119f9f05 dns_zone_name = oxide.test found_so_far = 1 version = 2
1032023-12-12T05:46:02.056ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
1042023-12-12T05:46:02.057ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2023-12-12 05:43:20.566607 UTC dns_group = external generation = 2
1052023-12-12T05:46:02.057ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 19.965004ms iteration = 6
1062023-12-12T05:46:02.060ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2023-12-12T05:43:02.185336Z, time_modified: 2023-12-12T05:43:02.185336Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 3b6a6fc1-cb1d-4a52-8072-dd8e5d15f389, db_silo: Silo { identity: SiloIdentity { id: 3b6a6fc1-cb1d-4a52-8072-dd8e5d15f389, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2023-12-12T05:43:20.510842Z, time_modified: 2023-12-12T05:43:20.510842Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "09cd4777b7e66ecc555780ddf2e4364cfefa9836047f5f5d63dfec10c20bcbfe" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 3b6a6fc1-cb1d-4a52-8072-dd8e5d15f389, db_silo: Silo { identity: SiloIdentity { id: 3b6a6fc1-cb1d-4a52-8072-dd8e5d15f389, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2023-12-12T05:43:20.510842Z, time_modified: 2023-12-12T05:43:20.510842Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "09cd4777b7e66ecc555780ddf2e4364cfefa9836047f5f5d63dfec10c20bcbfe" }] }) }
1072023-12-12T05:46:02.060ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 23.893631ms iteration = 6
1082023-12-12T05:46:02.085ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
1092023-12-12T05:46:02.085ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
1102023-12-12T05:46:02.085ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
1112023-12-12T05:46:02.085ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
1122023-12-12T05:46:02.085ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
1132023-12-12T05:46:02.086ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1142023-12-12T05:46:02.089ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
1152023-12-12T05:46:02.089ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
1162023-12-12T05:46:02.089ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
1172023-12-12T05:46:02.090ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
1182023-12-12T05:46:02.090ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = aa66fdf1-2426-44d7-a757-e029895906aa dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
1192023-12-12T05:46:02.090ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
1202023-12-12T05:46:02.090ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
1212023-12-12T05:46:02.092ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1222023-12-12T05:46:02.094ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
1232023-12-12T05:46:02.094ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
1242023-12-12T05:46:02.095ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
1252023-12-12T05:46:02.097ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = aa66fdf1-2426-44d7-a757-e029895906aa dns_zone_name = control-plane.oxide.internal found_so_far = 51 version = 1
1262023-12-12T05:46:02.097ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
1272023-12-12T05:46:02.097ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2023-12-12 05:43:20.140927 UTC dns_group = internal generation = 1
1282023-12-12T05:46:02.097ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 60.254008ms iteration = 6
1292023-12-12T05:46:02.105ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
1302023-12-12T05:46:02.105ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
1312023-12-12T05:46:02.165ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
1322023-12-12T05:46:02.166ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
1332023-12-12T05:46:02.166ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::8)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "6f438fe2-df46-42e4-b4c5-5642a5b1fdf9", "date": "Tue, 12 Dec 2023 05:46:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
1342023-12-12T05:46:02.166ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/dns_propagation.rs:173 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
1352023-12-12T05:46:02.168ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::9)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "d8cf8fcc-8f4a-40ad-beec-927bc2fc8968", "date": "Tue, 12 Dec 2023 05:46:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
1362023-12-12T05:46:02.168ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/dns_propagation.rs:173 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
1372023-12-12T05:46:02.169ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/dns_propagation.rs:110 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
1382023-12-12T05:46:02.169ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 132.788111ms iteration = 8
1392023-12-12T05:46:02.196ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
1402023-12-12T05:46:02.196ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:3::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "19b7082c-c2a3-4ff3-8580-443566c9da34", "date": "Tue, 12 Dec 2023 05:46:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
1412023-12-12T05:46:02.196ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/dns_propagation.rs:173 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
1422023-12-12T05:46:02.198ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:2::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "e9626b8e-533b-406b-a9cf-f365a46a4352", "date": "Tue, 12 Dec 2023 05:46:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
1432023-12-12T05:46:02.198ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/dns_propagation.rs:173 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
1442023-12-12T05:46:02.199ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:1::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "ffc4bf6c-40df-43c4-8f6b-1c65188f04f3", "date": "Tue, 12 Dec 2023 05:46:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
1452023-12-12T05:46:02.199ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/dns_propagation.rs:173 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
1462023-12-12T05:46:02.200ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/dns_propagation.rs:110 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
1472023-12-12T05:46:02.200ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 163.533513ms iteration = 8
1482023-12-12T05:46:02.281ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 514 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:36542 req_id = ed50119b-eb74-4118-bff0-180a5db8b95a response_code = 200 uri = /metrics/collect/47e44ab6-ee0b-4407-95b8-c3af199399c8
1492023-12-12T05:46:12.281ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 488 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:36542 req_id = 85d540a2-f83c-4f65-aa74-382c6a0b6fb6 response_code = 200 uri = /metrics/collect/47e44ab6-ee0b-4407-95b8-c3af199399c8
1502023-12-12T05:46:22.281ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 808 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:36542 req_id = 417527ec-5637-4b2d-9db3-f81b56fa5d67 response_code = 200 uri = /metrics/collect/47e44ab6-ee0b-4407-95b8-c3af199399c8
1512023-12-12T05:46:32.039ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 9 reason = Timeout
1522023-12-12T05:46:32.039ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 9 reason = Timeout
1532023-12-12T05:46:32.039ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
1542023-12-12T05:46:32.039ZWARNnexus (BackgroundTasks): phantom disk task started background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:50
1552023-12-12T05:46:32.039ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
1562023-12-12T05:46:32.041ZDEBGnexus (DpdClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
1572023-12-12T05:46:32.041ZDEBGnexus (DpdClient): client response result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "e7a1cf1a-3aa8-4ef5-94d1-f2f908771a21", "content-length": "1", "date": "Tue, 12 Dec 2023 05:46:31 GMT"} })
1582023-12-12T05:46:32.041ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
1592023-12-12T05:46:32.041ZWARNnexus (BackgroundTasks): phantom disk task done background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:96
1602023-12-12T05:46:32.041ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 2.337167ms iteration = 9
1612023-12-12T05:46:32.042ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
1622023-12-12T05:46:32.043ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 3.824116ms iteration = 9
1632023-12-12T05:46:32.281ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 410 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:36542 req_id = 56cdeee0-76b7-4fa4-8574-740dd29598f5 response_code = 200 uri = /metrics/collect/47e44ab6-ee0b-4407-95b8-c3af199399c8
1642023-12-12T05:46:42.281ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 394 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:36542 req_id = 9c658f31-9a36-4714-a677-3f314da19697 response_code = 200 uri = /metrics/collect/47e44ab6-ee0b-4407-95b8-c3af199399c8
1652023-12-12T05:46:52.282ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 406 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:36542 req_id = 9159255f-6ab9-4c8c-9ff7-479bf1384b5c response_code = 200 uri = /metrics/collect/47e44ab6-ee0b-4407-95b8-c3af199399c8
1662023-12-12T05:47:02.038ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 9 reason = Timeout
1672023-12-12T05:47:02.038ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 7 reason = Timeout
1682023-12-12T05:47:02.038ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
1692023-12-12T05:47:02.038ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
1702023-12-12T05:47:02.038ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 7 reason = Timeout
1712023-12-12T05:47:02.038ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 7 reason = Timeout
1722023-12-12T05:47:02.038ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_external dns_group = external roles = RoleSet { roles: {} }
1732023-12-12T05:47:02.038ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_internal dns_group = internal roles = RoleSet { roles: {} }
1742023-12-12T05:47:02.038ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 7 reason = Timeout
1752023-12-12T05:47:02.038ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 9 reason = Timeout
1762023-12-12T05:47:02.038ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
1772023-12-12T05:47:02.038ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
1782023-12-12T05:47:02.038ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 7 reason = Timeout
1792023-12-12T05:47:02.038ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
1802023-12-12T05:47:02.039ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
1812023-12-12T05:47:02.039ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_internal dns_group = internal resource = Database result = Ok(())
1822023-12-12T05:47:02.039ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_external dns_group = external resource = Database result = Ok(())
1832023-12-12T05:47:02.039ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
1842023-12-12T05:47:02.039ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
1852023-12-12T05:47:02.039ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
1862023-12-12T05:47:02.040ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
1872023-12-12T05:47:02.040ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
1882023-12-12T05:47:02.040ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 10 reason = Timeout
1892023-12-12T05:47:02.040ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 10 reason = Timeout
1902023-12-12T05:47:02.040ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
1912023-12-12T05:47:02.040ZWARNnexus (BackgroundTasks): phantom disk task started background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:50
1922023-12-12T05:47:02.040ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
1932023-12-12T05:47:02.041ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1942023-12-12T05:47:02.041ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
1952023-12-12T05:47:02.041ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1962023-12-12T05:47:02.042ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1972023-12-12T05:47:02.042ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1982023-12-12T05:47:02.042ZDEBGnexus (DpdClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
1992023-12-12T05:47:02.042ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2002023-12-12T05:47:02.043ZDEBGnexus (DpdClient): client response result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "0e700990-a885-45e2-a010-4c30589bee3c", "content-length": "1", "date": "Tue, 12 Dec 2023 05:47:01 GMT"} })
2012023-12-12T05:47:02.043ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
2022023-12-12T05:47:02.043ZWARNnexus (BackgroundTasks): phantom disk task done background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:96
2032023-12-12T05:47:02.043ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 2.904015ms iteration = 10
2042023-12-12T05:47:02.043ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_internal dns_group = internal resource = Fleet result = Ok(())
2052023-12-12T05:47:02.043ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_internal dns_group = internal roles = RoleSet { roles: {} }
2062023-12-12T05:47:02.044ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_internal dns_group = internal resource = Database result = Ok(())
2072023-12-12T05:47:02.044ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
2082023-12-12T05:47:02.045ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
2092023-12-12T05:47:02.045ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
2102023-12-12T05:47:02.045ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
2112023-12-12T05:47:02.045ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 4.962009ms iteration = 10
2122023-12-12T05:47:02.046ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } dns_group = internal
2132023-12-12T05:47:02.046ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 7.423468ms iteration = 7
2142023-12-12T05:47:02.046ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
2152023-12-12T05:47:02.046ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_external dns_group = external resource = Fleet result = Ok(())
2162023-12-12T05:47:02.046ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_external dns_group = external roles = RoleSet { roles: {} }
2172023-12-12T05:47:02.046ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_external dns_group = external resource = Database result = Ok(())
2182023-12-12T05:47:02.046ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
2192023-12-12T05:47:02.047ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
2202023-12-12T05:47:02.047ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
2212023-12-12T05:47:02.047ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
2222023-12-12T05:47:02.047ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
2232023-12-12T05:47:02.047ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
2242023-12-12T05:47:02.047ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
2252023-12-12T05:47:02.047ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
2262023-12-12T05:47:02.047ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
2272023-12-12T05:47:02.047ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
2282023-12-12T05:47:02.047ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
2292023-12-12T05:47:02.047ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
2302023-12-12T05:47:02.047ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
2312023-12-12T05:47:02.048ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2322023-12-12T05:47:02.048ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2332023-12-12T05:47:02.049ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2342023-12-12T05:47:02.053ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
2352023-12-12T05:47:02.053ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
2362023-12-12T05:47:02.053ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
2372023-12-12T05:47:02.053ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
2382023-12-12T05:47:02.054ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
2392023-12-12T05:47:02.054ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
2402023-12-12T05:47:02.054ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
2412023-12-12T05:47:02.054ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
2422023-12-12T05:47:02.054ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
2432023-12-12T05:47:02.057ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
2442023-12-12T05:47:02.057ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
2452023-12-12T05:47:02.060ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
2462023-12-12T05:47:02.060ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = aa66fdf1-2426-44d7-a757-e029895906aa dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
2472023-12-12T05:47:02.060ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
2482023-12-12T05:47:02.060ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
2492023-12-12T05:47:02.063ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
2502023-12-12T05:47:02.063ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 54229ede-970c-45c1-9159-c834119f9f05 dns_zone_name = oxide.test found_so_far = 0 version = 2
2512023-12-12T05:47:02.063ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
2522023-12-12T05:47:02.063ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
2532023-12-12T05:47:02.064ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2542023-12-12T05:47:02.064ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2552023-12-12T05:47:02.065ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2562023-12-12T05:47:02.067ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
2572023-12-12T05:47:02.067ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
2582023-12-12T05:47:02.067ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
2592023-12-12T05:47:02.068ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
2602023-12-12T05:47:02.069ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
2612023-12-12T05:47:02.069ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
2622023-12-12T05:47:02.069ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
2632023-12-12T05:47:02.069ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
2642023-12-12T05:47:02.070ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
2652023-12-12T05:47:02.071ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = aa66fdf1-2426-44d7-a757-e029895906aa dns_zone_name = control-plane.oxide.internal found_so_far = 51 version = 1
2662023-12-12T05:47:02.071ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
2672023-12-12T05:47:02.071ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2023-12-12 05:43:20.140927 UTC dns_group = internal generation = 1
2682023-12-12T05:47:02.071ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 32.381041ms iteration = 7
2692023-12-12T05:47:02.071ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 54229ede-970c-45c1-9159-c834119f9f05 dns_zone_name = oxide.test found_so_far = 1 version = 2
2702023-12-12T05:47:02.071ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
2712023-12-12T05:47:02.071ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2023-12-12 05:43:20.566607 UTC dns_group = external generation = 2
2722023-12-12T05:47:02.071ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 33.251561ms iteration = 7
2732023-12-12T05:47:02.079ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2023-12-12T05:43:02.185336Z, time_modified: 2023-12-12T05:43:02.185336Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 3b6a6fc1-cb1d-4a52-8072-dd8e5d15f389, db_silo: Silo { identity: SiloIdentity { id: 3b6a6fc1-cb1d-4a52-8072-dd8e5d15f389, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2023-12-12T05:43:20.510842Z, time_modified: 2023-12-12T05:43:20.510842Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "09cd4777b7e66ecc555780ddf2e4364cfefa9836047f5f5d63dfec10c20bcbfe" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 3b6a6fc1-cb1d-4a52-8072-dd8e5d15f389, db_silo: Silo { identity: SiloIdentity { id: 3b6a6fc1-cb1d-4a52-8072-dd8e5d15f389, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2023-12-12T05:43:20.510842Z, time_modified: 2023-12-12T05:43:20.510842Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "09cd4777b7e66ecc555780ddf2e4364cfefa9836047f5f5d63dfec10c20bcbfe" }] }) }
2742023-12-12T05:47:02.079ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 40.475926ms iteration = 7
2752023-12-12T05:47:02.083ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
2762023-12-12T05:47:02.087ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
2772023-12-12T05:47:02.090ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } dns_group = external
2782023-12-12T05:47:02.090ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 52.114433ms iteration = 7
2792023-12-12T05:47:02.121ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
2802023-12-12T05:47:02.121ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::8)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "bd63f147-6cf6-4de8-97c4-0df004363d99", "date": "Tue, 12 Dec 2023 05:47:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
2812023-12-12T05:47:02.121ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/dns_propagation.rs:173 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
2822023-12-12T05:47:02.123ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::9)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "f0d29d3a-590d-41f5-848f-d65bfb03fd59", "date": "Tue, 12 Dec 2023 05:47:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
2832023-12-12T05:47:02.123ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/dns_propagation.rs:173 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
2842023-12-12T05:47:02.123ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/dns_propagation.rs:110 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
2852023-12-12T05:47:02.123ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 85.081101ms iteration = 9
2862023-12-12T05:47:02.124ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
2872023-12-12T05:47:02.155ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
2882023-12-12T05:47:02.156ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:3::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "272af2ea-b837-4c11-badf-5bb0fdedad0e", "date": "Tue, 12 Dec 2023 05:47:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
2892023-12-12T05:47:02.156ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/dns_propagation.rs:173 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
2902023-12-12T05:47:02.158ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:2::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "888741c1-aa64-4815-889f-9a622b8f4d59", "date": "Tue, 12 Dec 2023 05:47:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
2912023-12-12T05:47:02.158ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/dns_propagation.rs:173 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
2922023-12-12T05:47:02.159ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:1::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "5b0dfadc-fcd5-4466-b570-ea353fca1acc", "date": "Tue, 12 Dec 2023 05:47:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
2932023-12-12T05:47:02.159ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/dns_propagation.rs:173 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
2942023-12-12T05:47:02.160ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/dns_propagation.rs:110 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
2952023-12-12T05:47:02.160ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 121.886464ms iteration = 9
2962023-12-12T05:47:02.281ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 545 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:36542 req_id = c22f3a86-5fcb-49a2-9837-eef3d94f548b response_code = 200 uri = /metrics/collect/47e44ab6-ee0b-4407-95b8-c3af199399c8
2972023-12-12T05:47:12.282ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 490 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:36542 req_id = fe9cbc9a-a37f-42da-a216-d6f9aa2ff35f response_code = 200 uri = /metrics/collect/47e44ab6-ee0b-4407-95b8-c3af199399c8
2982023-12-12T05:47:22.283ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 494 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:36542 req_id = cf90020a-77b4-40a2-8c3d-ad2250fdaad1 response_code = 200 uri = /metrics/collect/47e44ab6-ee0b-4407-95b8-c3af199399c8
2992023-12-12T05:47:32.041ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 11 reason = Timeout
3002023-12-12T05:47:32.041ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
3012023-12-12T05:47:32.041ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 11 reason = Timeout
3022023-12-12T05:47:32.041ZWARNnexus (BackgroundTasks): phantom disk task started background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:50
3032023-12-12T05:47:32.042ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
3042023-12-12T05:47:32.043ZDEBGnexus (DpdClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
3052023-12-12T05:47:32.043ZDEBGnexus (DpdClient): client response result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "33cc7392-cc18-4d96-8a1f-b1108f93c27d", "content-length": "1", "date": "Tue, 12 Dec 2023 05:47:31 GMT"} })
3062023-12-12T05:47:32.043ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
3072023-12-12T05:47:32.043ZWARNnexus (BackgroundTasks): phantom disk task done background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:96
3082023-12-12T05:47:32.043ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 2.011354ms iteration = 11
3092023-12-12T05:47:32.044ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
3102023-12-12T05:47:32.045ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 3.43277ms iteration = 11
3112023-12-12T05:47:32.283ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 487 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:36542 req_id = 0bd18290-9056-4b65-b838-2f60aede6971 response_code = 200 uri = /metrics/collect/47e44ab6-ee0b-4407-95b8-c3af199399c8
3122023-12-12T05:47:42.282ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 416 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:36542 req_id = 4617d2ce-09f9-4adc-9c55-6742d3523f40 response_code = 200 uri = /metrics/collect/47e44ab6-ee0b-4407-95b8-c3af199399c8
3132023-12-12T05:47:52.283ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 493 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:36542 req_id = 2dc79fc7-335c-4c5a-b210-138460a0fc63 response_code = 200 uri = /metrics/collect/47e44ab6-ee0b-4407-95b8-c3af199399c8
3142023-12-12T05:48:02.039ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 10 reason = Timeout
3152023-12-12T05:48:02.039ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 8 reason = Timeout
3162023-12-12T05:48:02.039ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
3172023-12-12T05:48:02.039ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
3182023-12-12T05:48:02.039ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 8 reason = Timeout
3192023-12-12T05:48:02.039ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 8 reason = Timeout
3202023-12-12T05:48:02.039ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
3212023-12-12T05:48:02.039ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
3222023-12-12T05:48:02.039ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_internal dns_group = internal roles = RoleSet { roles: {} }
3232023-12-12T05:48:02.039ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 8 reason = Timeout
3242023-12-12T05:48:02.039ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 8 reason = Timeout
3252023-12-12T05:48:02.039ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
3262023-12-12T05:48:02.039ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 10 reason = Timeout
3272023-12-12T05:48:02.039ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_external dns_group = external roles = RoleSet { roles: {} }
3282023-12-12T05:48:02.039ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_internal dns_group = internal resource = Database result = Ok(())
3292023-12-12T05:48:02.039ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
3302023-12-12T05:48:02.039ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
3312023-12-12T05:48:02.039ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
3322023-12-12T05:48:02.039ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_external dns_group = external resource = Database result = Ok(())
3332023-12-12T05:48:02.040ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
3342023-12-12T05:48:02.040ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
3352023-12-12T05:48:02.041ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
3362023-12-12T05:48:02.041ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
3372023-12-12T05:48:02.041ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 12 reason = Timeout
3382023-12-12T05:48:02.041ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 12 reason = Timeout
3392023-12-12T05:48:02.041ZWARNnexus (BackgroundTasks): phantom disk task started background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:50
3402023-12-12T05:48:02.041ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
3412023-12-12T05:48:02.041ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3422023-12-12T05:48:02.041ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
3432023-12-12T05:48:02.041ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3442023-12-12T05:48:02.042ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3452023-12-12T05:48:02.042ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3462023-12-12T05:48:02.042ZDEBGnexus (DpdClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
3472023-12-12T05:48:02.043ZDEBGnexus (DpdClient): client response result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "5d05a516-3fbc-468a-a07a-430b6c000d79", "content-length": "1", "date": "Tue, 12 Dec 2023 05:48:01 GMT"} })
3482023-12-12T05:48:02.043ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
3492023-12-12T05:48:02.043ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3502023-12-12T05:48:02.043ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
3512023-12-12T05:48:02.044ZWARNnexus (BackgroundTasks): phantom disk task done background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:96
3522023-12-12T05:48:02.044ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 2.666121ms iteration = 12
3532023-12-12T05:48:02.044ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_internal dns_group = internal resource = Fleet result = Ok(())
3542023-12-12T05:48:02.044ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_internal dns_group = internal roles = RoleSet { roles: {} }
3552023-12-12T05:48:02.044ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 3.42662ms iteration = 12
3562023-12-12T05:48:02.044ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_external dns_group = external resource = Fleet result = Ok(())
3572023-12-12T05:48:02.044ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_external dns_group = external roles = RoleSet { roles: {} }
3582023-12-12T05:48:02.044ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_internal dns_group = internal resource = Database result = Ok(())
3592023-12-12T05:48:02.045ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_external dns_group = external resource = Database result = Ok(())
3602023-12-12T05:48:02.046ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
3612023-12-12T05:48:02.046ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
3622023-12-12T05:48:02.046ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
3632023-12-12T05:48:02.046ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } dns_group = internal
3642023-12-12T05:48:02.046ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 7.394897ms iteration = 8
3652023-12-12T05:48:02.046ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
3662023-12-12T05:48:02.046ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } dns_group = external
3672023-12-12T05:48:02.046ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 7.497728ms iteration = 8
3682023-12-12T05:48:02.047ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
3692023-12-12T05:48:02.047ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
3702023-12-12T05:48:02.047ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
3712023-12-12T05:48:02.047ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
3722023-12-12T05:48:02.047ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
3732023-12-12T05:48:02.047ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
3742023-12-12T05:48:02.047ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
3752023-12-12T05:48:02.047ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
3762023-12-12T05:48:02.048ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
3772023-12-12T05:48:02.048ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
3782023-12-12T05:48:02.048ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
3792023-12-12T05:48:02.048ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
3802023-12-12T05:48:02.048ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
3812023-12-12T05:48:02.048ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3822023-12-12T05:48:02.049ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3832023-12-12T05:48:02.049ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3842023-12-12T05:48:02.052ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
3852023-12-12T05:48:02.052ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
3862023-12-12T05:48:02.052ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
3872023-12-12T05:48:02.052ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
3882023-12-12T05:48:02.052ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
3892023-12-12T05:48:02.053ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
3902023-12-12T05:48:02.053ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
3912023-12-12T05:48:02.053ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
3922023-12-12T05:48:02.053ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
3932023-12-12T05:48:02.053ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
3942023-12-12T05:48:02.054ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
3952023-12-12T05:48:02.054ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
3962023-12-12T05:48:02.054ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 54229ede-970c-45c1-9159-c834119f9f05 dns_zone_name = oxide.test found_so_far = 0 version = 2
3972023-12-12T05:48:02.054ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
3982023-12-12T05:48:02.054ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
3992023-12-12T05:48:02.054ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
4002023-12-12T05:48:02.054ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = aa66fdf1-2426-44d7-a757-e029895906aa dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
4012023-12-12T05:48:02.054ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
4022023-12-12T05:48:02.054ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
4032023-12-12T05:48:02.055ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4042023-12-12T05:48:02.055ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4052023-12-12T05:48:02.055ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4062023-12-12T05:48:02.057ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
4072023-12-12T05:48:02.057ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
4082023-12-12T05:48:02.057ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
4092023-12-12T05:48:02.059ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
4102023-12-12T05:48:02.059ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
4112023-12-12T05:48:02.059ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
4122023-12-12T05:48:02.059ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
4132023-12-12T05:48:02.059ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
4142023-12-12T05:48:02.059ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
4152023-12-12T05:48:02.063ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 54229ede-970c-45c1-9159-c834119f9f05 dns_zone_name = oxide.test found_so_far = 1 version = 2
4162023-12-12T05:48:02.063ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
4172023-12-12T05:48:02.063ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2023-12-12 05:43:20.566607 UTC dns_group = external generation = 2
4182023-12-12T05:48:02.063ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 23.741789ms iteration = 8
4192023-12-12T05:48:02.067ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = aa66fdf1-2426-44d7-a757-e029895906aa dns_zone_name = control-plane.oxide.internal found_so_far = 51 version = 1
4202023-12-12T05:48:02.067ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
4212023-12-12T05:48:02.067ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2023-12-12 05:43:20.140927 UTC dns_group = internal generation = 1
4222023-12-12T05:48:02.067ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 28.292243ms iteration = 8
4232023-12-12T05:48:02.069ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2023-12-12T05:43:02.185336Z, time_modified: 2023-12-12T05:43:02.185336Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 3b6a6fc1-cb1d-4a52-8072-dd8e5d15f389, db_silo: Silo { identity: SiloIdentity { id: 3b6a6fc1-cb1d-4a52-8072-dd8e5d15f389, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2023-12-12T05:43:20.510842Z, time_modified: 2023-12-12T05:43:20.510842Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "09cd4777b7e66ecc555780ddf2e4364cfefa9836047f5f5d63dfec10c20bcbfe" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 3b6a6fc1-cb1d-4a52-8072-dd8e5d15f389, db_silo: Silo { identity: SiloIdentity { id: 3b6a6fc1-cb1d-4a52-8072-dd8e5d15f389, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2023-12-12T05:43:20.510842Z, time_modified: 2023-12-12T05:43:20.510842Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "09cd4777b7e66ecc555780ddf2e4364cfefa9836047f5f5d63dfec10c20bcbfe" }] }) }
4242023-12-12T05:48:02.069ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 30.62722ms iteration = 8
4252023-12-12T05:48:02.087ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
4262023-12-12T05:48:02.090ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
4272023-12-12T05:48:02.124ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
4282023-12-12T05:48:02.128ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
4292023-12-12T05:48:02.129ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::9)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "d3e21c9f-0e90-4046-8341-a473d6ab577f", "date": "Tue, 12 Dec 2023 05:48:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
4302023-12-12T05:48:02.129ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/dns_propagation.rs:173 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
4312023-12-12T05:48:02.130ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::8)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "fdbe5e8b-64e1-4cae-9520-751372338b19", "date": "Tue, 12 Dec 2023 05:48:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
4322023-12-12T05:48:02.130ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/dns_propagation.rs:173 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
4332023-12-12T05:48:02.131ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/dns_propagation.rs:110 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
4342023-12-12T05:48:02.131ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 91.887191ms iteration = 10
4352023-12-12T05:48:02.155ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
4362023-12-12T05:48:02.156ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:1::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "c64aebb3-040d-4473-a511-4c28cc4b4684", "date": "Tue, 12 Dec 2023 05:48:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
4372023-12-12T05:48:02.156ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/dns_propagation.rs:173 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
4382023-12-12T05:48:02.157ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:3::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "9c5c0a67-60e6-4aa4-8c04-c8dabd5025be", "date": "Tue, 12 Dec 2023 05:48:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
4392023-12-12T05:48:02.157ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/dns_propagation.rs:173 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
4402023-12-12T05:48:02.157ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:2::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "8440d62a-8b99-44db-bb52-25cd43e648ef", "date": "Tue, 12 Dec 2023 05:48:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
4412023-12-12T05:48:02.157ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/dns_propagation.rs:173 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
4422023-12-12T05:48:02.158ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/dns_propagation.rs:110 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
4432023-12-12T05:48:02.158ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 119.215472ms iteration = 10
4442023-12-12T05:48:02.283ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 582 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:36542 req_id = 28324d2b-d9b9-45c0-85ab-beb05e5526fd response_code = 200 uri = /metrics/collect/47e44ab6-ee0b-4407-95b8-c3af199399c8
4452023-12-12T05:48:12.282ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 708 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:36542 req_id = 650c779e-03bd-4368-b939-f7b106db6a03 response_code = 200 uri = /metrics/collect/47e44ab6-ee0b-4407-95b8-c3af199399c8
4462023-12-12T05:48:22.283ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 596 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:36542 req_id = 38c865d3-34c1-447f-8934-078ba7592186 response_code = 200 uri = /metrics/collect/47e44ab6-ee0b-4407-95b8-c3af199399c8
4472023-12-12T05:48:32.041ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 13 reason = Timeout
4482023-12-12T05:48:32.041ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 13 reason = Timeout
4492023-12-12T05:48:32.041ZWARNnexus (BackgroundTasks): phantom disk task started background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:50
4502023-12-12T05:48:32.041ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
4512023-12-12T05:48:32.042ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
4522023-12-12T05:48:32.043ZDEBGnexus (DpdClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
4532023-12-12T05:48:32.043ZDEBGnexus (DpdClient): client response result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "0e5625d6-c326-423d-9469-e20b0e4167f2", "content-length": "1", "date": "Tue, 12 Dec 2023 05:48:31 GMT"} })
4542023-12-12T05:48:32.043ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
4552023-12-12T05:48:32.043ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
4562023-12-12T05:48:32.044ZWARNnexus (BackgroundTasks): phantom disk task done background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:96
4572023-12-12T05:48:32.044ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 2.55676ms iteration = 13
4582023-12-12T05:48:32.044ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 3.162457ms iteration = 13
4592023-12-12T05:48:32.284ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 812 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:36542 req_id = 8654aede-24cc-4390-b2b4-eddaebdf02e5 response_code = 200 uri = /metrics/collect/47e44ab6-ee0b-4407-95b8-c3af199399c8
4602023-12-12T05:48:42.282ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 459 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:36542 req_id = 156c1fe3-5312-4d82-b8e8-2a52e848539c response_code = 200 uri = /metrics/collect/47e44ab6-ee0b-4407-95b8-c3af199399c8
4612023-12-12T05:48:52.283ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 1073 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:36542 req_id = 181d3989-ae65-4853-b894-5c21fd3bfce4 response_code = 200 uri = /metrics/collect/47e44ab6-ee0b-4407-95b8-c3af199399c8
4622023-12-12T05:49:02.040ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 11 reason = Timeout
4632023-12-12T05:49:02.040ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 9 reason = Timeout
4642023-12-12T05:49:02.040ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
4652023-12-12T05:49:02.040ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
4662023-12-12T05:49:02.040ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 9 reason = Timeout
4672023-12-12T05:49:02.040ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
4682023-12-12T05:49:02.040ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 11 reason = Timeout
4692023-12-12T05:49:02.040ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 9 reason = Timeout
4702023-12-12T05:49:02.040ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_internal dns_group = internal roles = RoleSet { roles: {} }
4712023-12-12T05:49:02.040ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 9 reason = Timeout
4722023-12-12T05:49:02.040ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 9 reason = Timeout
4732023-12-12T05:49:02.040ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
4742023-12-12T05:49:02.040ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_external dns_group = external roles = RoleSet { roles: {} }
4752023-12-12T05:49:02.040ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
4762023-12-12T05:49:02.040ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
4772023-12-12T05:49:02.040ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
4782023-12-12T05:49:02.040ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_internal dns_group = internal resource = Database result = Ok(())
4792023-12-12T05:49:02.040ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_external dns_group = external resource = Database result = Ok(())
4802023-12-12T05:49:02.041ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
4812023-12-12T05:49:02.041ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
4822023-12-12T05:49:02.041ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
4832023-12-12T05:49:02.041ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
4842023-12-12T05:49:02.041ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
4852023-12-12T05:49:02.042ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4862023-12-12T05:49:02.042ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 14 reason = Timeout
4872023-12-12T05:49:02.042ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
4882023-12-12T05:49:02.042ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 14 reason = Timeout
4892023-12-12T05:49:02.042ZWARNnexus (BackgroundTasks): phantom disk task started background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:50
4902023-12-12T05:49:02.042ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
4912023-12-12T05:49:02.042ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4922023-12-12T05:49:02.043ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4932023-12-12T05:49:02.043ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4942023-12-12T05:49:02.043ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4952023-12-12T05:49:02.044ZDEBGnexus (DpdClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
4962023-12-12T05:49:02.045ZDEBGnexus (DpdClient): client response result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "42e6ddae-4e9b-4f2f-b31d-7d150d4bc9f8", "content-length": "1", "date": "Tue, 12 Dec 2023 05:49:02 GMT"} })
4972023-12-12T05:49:02.045ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
4982023-12-12T05:49:02.045ZWARNnexus (BackgroundTasks): phantom disk task done background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:96
4992023-12-12T05:49:02.045ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 2.928195ms iteration = 14
5002023-12-12T05:49:02.045ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
5012023-12-12T05:49:02.046ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
5022023-12-12T05:49:02.046ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
5032023-12-12T05:49:02.046ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 4.445372ms iteration = 14
5042023-12-12T05:49:02.047ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
5052023-12-12T05:49:02.047ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_external dns_group = external resource = Fleet result = Ok(())
5062023-12-12T05:49:02.047ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_external dns_group = external roles = RoleSet { roles: {} }
5072023-12-12T05:49:02.047ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_internal dns_group = internal resource = Fleet result = Ok(())
5082023-12-12T05:49:02.047ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_internal dns_group = internal roles = RoleSet { roles: {} }
5092023-12-12T05:49:02.047ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_external dns_group = external resource = Database result = Ok(())
5102023-12-12T05:49:02.047ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
5112023-12-12T05:49:02.047ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
5122023-12-12T05:49:02.047ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_internal dns_group = internal resource = Database result = Ok(())
5132023-12-12T05:49:02.048ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
5142023-12-12T05:49:02.048ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
5152023-12-12T05:49:02.048ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
5162023-12-12T05:49:02.049ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
5172023-12-12T05:49:02.049ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
5182023-12-12T05:49:02.049ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
5192023-12-12T05:49:02.049ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
5202023-12-12T05:49:02.049ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
5212023-12-12T05:49:02.049ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
5222023-12-12T05:49:02.049ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
5232023-12-12T05:49:02.049ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
5242023-12-12T05:49:02.049ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
5252023-12-12T05:49:02.049ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5262023-12-12T05:49:02.050ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } dns_group = external
5272023-12-12T05:49:02.050ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 9.621004ms iteration = 9
5282023-12-12T05:49:02.050ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5292023-12-12T05:49:02.050ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } dns_group = internal
5302023-12-12T05:49:02.050ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 10.254421ms iteration = 9
5312023-12-12T05:49:02.050ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5322023-12-12T05:49:02.053ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
5332023-12-12T05:49:02.053ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
5342023-12-12T05:49:02.053ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
5352023-12-12T05:49:02.053ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
5362023-12-12T05:49:02.053ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
5372023-12-12T05:49:02.054ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
5382023-12-12T05:49:02.054ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
5392023-12-12T05:49:02.054ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
5402023-12-12T05:49:02.054ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
5412023-12-12T05:49:02.054ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
5422023-12-12T05:49:02.054ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
5432023-12-12T05:49:02.054ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
5442023-12-12T05:49:02.054ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = aa66fdf1-2426-44d7-a757-e029895906aa dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
5452023-12-12T05:49:02.054ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
5462023-12-12T05:49:02.055ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
5472023-12-12T05:49:02.055ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
5482023-12-12T05:49:02.055ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 54229ede-970c-45c1-9159-c834119f9f05 dns_zone_name = oxide.test found_so_far = 0 version = 2
5492023-12-12T05:49:02.055ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
5502023-12-12T05:49:02.055ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
5512023-12-12T05:49:02.056ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5522023-12-12T05:49:02.056ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5532023-12-12T05:49:02.056ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5542023-12-12T05:49:02.058ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
5552023-12-12T05:49:02.058ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
5562023-12-12T05:49:02.058ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
5572023-12-12T05:49:02.060ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
5582023-12-12T05:49:02.060ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
5592023-12-12T05:49:02.060ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
5602023-12-12T05:49:02.060ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
5612023-12-12T05:49:02.060ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
5622023-12-12T05:49:02.061ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
5632023-12-12T05:49:02.061ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 54229ede-970c-45c1-9159-c834119f9f05 dns_zone_name = oxide.test found_so_far = 1 version = 2
5642023-12-12T05:49:02.061ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
5652023-12-12T05:49:02.061ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2023-12-12 05:43:20.566607 UTC dns_group = external generation = 2
5662023-12-12T05:49:02.061ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 21.572474ms iteration = 9
5672023-12-12T05:49:02.062ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = aa66fdf1-2426-44d7-a757-e029895906aa dns_zone_name = control-plane.oxide.internal found_so_far = 51 version = 1
5682023-12-12T05:49:02.062ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
5692023-12-12T05:49:02.062ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2023-12-12 05:43:20.140927 UTC dns_group = internal generation = 1
5702023-12-12T05:49:02.062ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 22.333312ms iteration = 9
5712023-12-12T05:49:02.065ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2023-12-12T05:43:02.185336Z, time_modified: 2023-12-12T05:43:02.185336Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 3b6a6fc1-cb1d-4a52-8072-dd8e5d15f389, db_silo: Silo { identity: SiloIdentity { id: 3b6a6fc1-cb1d-4a52-8072-dd8e5d15f389, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2023-12-12T05:43:20.510842Z, time_modified: 2023-12-12T05:43:20.510842Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "09cd4777b7e66ecc555780ddf2e4364cfefa9836047f5f5d63dfec10c20bcbfe" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 3b6a6fc1-cb1d-4a52-8072-dd8e5d15f389, db_silo: Silo { identity: SiloIdentity { id: 3b6a6fc1-cb1d-4a52-8072-dd8e5d15f389, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2023-12-12T05:43:20.510842Z, time_modified: 2023-12-12T05:43:20.510842Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "09cd4777b7e66ecc555780ddf2e4364cfefa9836047f5f5d63dfec10c20bcbfe" }] }) }
5722023-12-12T05:49:02.065ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 25.057635ms iteration = 9
5732023-12-12T05:49:02.105ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
5742023-12-12T05:49:02.109ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
5752023-12-12T05:49:02.166ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
5762023-12-12T05:49:02.170ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
5772023-12-12T05:49:02.170ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::9)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "e0ab3f37-de04-4d47-bac6-62997c407475", "date": "Tue, 12 Dec 2023 05:49:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
5782023-12-12T05:49:02.170ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/dns_propagation.rs:173 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
5792023-12-12T05:49:02.172ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::8)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "c511df0e-c6ab-4636-8d95-398f4438fab2", "date": "Tue, 12 Dec 2023 05:49:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
5802023-12-12T05:49:02.172ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/dns_propagation.rs:173 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
5812023-12-12T05:49:02.173ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/dns_propagation.rs:110 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
5822023-12-12T05:49:02.173ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 133.184786ms iteration = 11
5832023-12-12T05:49:02.198ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
5842023-12-12T05:49:02.199ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:3::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "c1c88b01-3b05-4ab2-9876-3f9a4e1121a5", "date": "Tue, 12 Dec 2023 05:49:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
5852023-12-12T05:49:02.199ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/dns_propagation.rs:173 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
5862023-12-12T05:49:02.201ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:2::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "c5f3dbb7-0a7d-4e1c-928c-ea7eafe1bb2f", "date": "Tue, 12 Dec 2023 05:49:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
5872023-12-12T05:49:02.201ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/dns_propagation.rs:173 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
5882023-12-12T05:49:02.203ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:1::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "bb4d9f2d-1217-470a-9a96-f466c4991a4e", "date": "Tue, 12 Dec 2023 05:49:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
5892023-12-12T05:49:02.203ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/dns_propagation.rs:173 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
5902023-12-12T05:49:02.204ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/dns_propagation.rs:110 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
5912023-12-12T05:49:02.204ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 164.725387ms iteration = 11
5922023-12-12T05:49:02.283ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 475 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:36542 req_id = 24b31449-fc1a-402f-a0ac-8acf0801ea63 response_code = 200 uri = /metrics/collect/47e44ab6-ee0b-4407-95b8-c3af199399c8
5932023-12-12T05:49:12.284ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 1073 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:36542 req_id = 6795123d-2814-46dc-81f7-3cc49a639d24 response_code = 200 uri = /metrics/collect/47e44ab6-ee0b-4407-95b8-c3af199399c8
5942023-12-12T05:49:22.284ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 813 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:36542 req_id = e67b2e32-c9d8-499f-b0b9-33388f84908a response_code = 200 uri = /metrics/collect/47e44ab6-ee0b-4407-95b8-c3af199399c8
5952023-12-12T05:49:32.043ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 15 reason = Timeout
5962023-12-12T05:49:32.043ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
5972023-12-12T05:49:32.043ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 15 reason = Timeout
5982023-12-12T05:49:32.043ZWARNnexus (BackgroundTasks): phantom disk task started background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:50
5992023-12-12T05:49:32.043ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
6002023-12-12T05:49:32.044ZDEBGnexus (DpdClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
6012023-12-12T05:49:32.044ZDEBGnexus (DpdClient): client response result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "d12e5997-4666-4b9a-9432-08c1e00cbc3b", "content-length": "1", "date": "Tue, 12 Dec 2023 05:49:31 GMT"} })
6022023-12-12T05:49:32.044ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
6032023-12-12T05:49:32.045ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
6042023-12-12T05:49:32.045ZWARNnexus (BackgroundTasks): phantom disk task done background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:96
6052023-12-12T05:49:32.045ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 2.368868ms iteration = 15
6062023-12-12T05:49:32.046ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 3.40716ms iteration = 15
6072023-12-12T05:49:32.284ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 843 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:36542 req_id = d5949f06-b706-4117-b9b7-7ac568ba04db response_code = 200 uri = /metrics/collect/47e44ab6-ee0b-4407-95b8-c3af199399c8
6082023-12-12T05:49:42.284ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 574 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:36542 req_id = 456723c8-b2c7-4e99-8214-f7073e259647 response_code = 200 uri = /metrics/collect/47e44ab6-ee0b-4407-95b8-c3af199399c8
6092023-12-12T05:49:47.779ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:769 local_addr = [fd00:1122:3344:101::c]:12221 remote_addr = [fd00:1122:3344:101::1]:33705
6102023-12-12T05:49:47.786ZINFOnexus: received new runtime state from sled agent file = nexus/src/app/instance.rs:1315 instance_id = 953e7a82-4116-4de5-be0d-86445eadf394 instance_state = InstanceRuntimeState { propolis_id: Some(92388aab-2213-4c6b-8308-7fa9d8730c7c), dst_propolis_id: None, migration_id: None, gen: Generation(3), time_updated: 2023-12-12T05:49:39.966360524Z } propolis_id = 92388aab-2213-4c6b-8308-7fa9d8730c7c vmm_state = VmmRuntimeState { state: Starting, gen: Generation(2), time_updated: 2023-12-12T05:49:47.776312643Z }
6112023-12-12T05:49:47.788ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:33705 req_id = f83632d0-fd6f-46db-b149-825ef7bcdd98 roles = RoleSet { roles: {} } uri = /instances/953e7a82-4116-4de5-be0d-86445eadf394
6122023-12-12T05:49:47.788ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:33705 req_id = f83632d0-fd6f-46db-b149-825ef7bcdd98 resource = Database result = Ok(()) uri = /instances/953e7a82-4116-4de5-be0d-86445eadf394
6132023-12-12T05:49:47.795ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:33705 req_id = f83632d0-fd6f-46db-b149-825ef7bcdd98 roles = RoleSet { roles: {} } uri = /instances/953e7a82-4116-4de5-be0d-86445eadf394
6142023-12-12T05:49:47.795ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:33705 req_id = f83632d0-fd6f-46db-b149-825ef7bcdd98 resource = Database result = Ok(()) uri = /instances/953e7a82-4116-4de5-be0d-86445eadf394
6152023-12-12T05:49:47.797ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:33705 req_id = f83632d0-fd6f-46db-b149-825ef7bcdd98 roles = RoleSet { roles: {} } uri = /instances/953e7a82-4116-4de5-be0d-86445eadf394
6162023-12-12T05:49:47.797ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:33705 req_id = f83632d0-fd6f-46db-b149-825ef7bcdd98 resource = Database result = Ok(()) uri = /instances/953e7a82-4116-4de5-be0d-86445eadf394
6172023-12-12T05:49:47.798ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:33705 req_id = f83632d0-fd6f-46db-b149-825ef7bcdd98 roles = RoleSet { roles: {} } uri = /instances/953e7a82-4116-4de5-be0d-86445eadf394
6182023-12-12T05:49:47.799ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:33705 req_id = f83632d0-fd6f-46db-b149-825ef7bcdd98 resource = Database result = Ok(()) uri = /instances/953e7a82-4116-4de5-be0d-86445eadf394
6192023-12-12T05:49:47.800ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:33705 req_id = f83632d0-fd6f-46db-b149-825ef7bcdd98 roles = RoleSet { roles: {} } uri = /instances/953e7a82-4116-4de5-be0d-86445eadf394
6202023-12-12T05:49:47.800ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:33705 req_id = f83632d0-fd6f-46db-b149-825ef7bcdd98 resource = Database result = Ok(()) uri = /instances/953e7a82-4116-4de5-be0d-86445eadf394
6212023-12-12T05:49:47.805ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:33705 req_id = f83632d0-fd6f-46db-b149-825ef7bcdd98 roles = RoleSet { roles: {} } uri = /instances/953e7a82-4116-4de5-be0d-86445eadf394
6222023-12-12T05:49:47.805ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:33705 req_id = f83632d0-fd6f-46db-b149-825ef7bcdd98 resource = Database result = Ok(()) uri = /instances/953e7a82-4116-4de5-be0d-86445eadf394
6232023-12-12T05:49:47.808ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:33705 req_id = f83632d0-fd6f-46db-b149-825ef7bcdd98 roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } uri = /instances/953e7a82-4116-4de5-be0d-86445eadf394
6242023-12-12T05:49:47.811ZDEBGnexus (dropshot_internal): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:33705 req_id = f83632d0-fd6f-46db-b149-825ef7bcdd98 resource = Instance { parent: Project { parent: Silo { parent: Fleet, key: 3b6a6fc1-cb1d-4a52-8072-dd8e5d15f389, lookup_type: ById(3b6a6fc1-cb1d-4a52-8072-dd8e5d15f389) }, key: d5ed4b91-81cd-476b-8f8e-359627fd51d1, lookup_type: ById(d5ed4b91-81cd-476b-8f8e-359627fd51d1) }, key: 953e7a82-4116-4de5-be0d-86445eadf394, lookup_type: ById(953e7a82-4116-4de5-be0d-86445eadf394) } result = Ok(()) uri = /instances/953e7a82-4116-4de5-be0d-86445eadf394
6252023-12-12T05:49:47.811ZDEBGnexus: instance state generation already advanced, won't touch network config instance_id = 953e7a82-4116-4de5-be0d-86445eadf394
6262023-12-12T05:49:47.849ZINFOnexus: instance and vmm updated by sled agent file = nexus/src/app/instance.rs:1451 instance_id = 953e7a82-4116-4de5-be0d-86445eadf394 instance_updated = false propolis_id = 92388aab-2213-4c6b-8308-7fa9d8730c7c vmm_updated = true
6272023-12-12T05:49:47.849ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 70141 local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:33705 req_id = f83632d0-fd6f-46db-b149-825ef7bcdd98 response_code = 204 uri = /instances/953e7a82-4116-4de5-be0d-86445eadf394
6282023-12-12T05:49:47.849ZINFOnexus: received new runtime state from sled agent file = nexus/src/app/instance.rs:1315 instance_id = 953e7a82-4116-4de5-be0d-86445eadf394 instance_state = InstanceRuntimeState { propolis_id: Some(92388aab-2213-4c6b-8308-7fa9d8730c7c), dst_propolis_id: None, migration_id: None, gen: Generation(3), time_updated: 2023-12-12T05:49:39.966360524Z } propolis_id = 92388aab-2213-4c6b-8308-7fa9d8730c7c vmm_state = VmmRuntimeState { state: Running, gen: Generation(3), time_updated: 2023-12-12T05:49:47.849494694Z }
6292023-12-12T05:49:47.849ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:33705 req_id = 2321550b-90d4-45c8-b2ef-669d24eeb708 roles = RoleSet { roles: {} } uri = /instances/953e7a82-4116-4de5-be0d-86445eadf394
6302023-12-12T05:49:47.849ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:33705 req_id = 2321550b-90d4-45c8-b2ef-669d24eeb708 resource = Database result = Ok(()) uri = /instances/953e7a82-4116-4de5-be0d-86445eadf394
6312023-12-12T05:49:47.851ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:33705 req_id = 2321550b-90d4-45c8-b2ef-669d24eeb708 roles = RoleSet { roles: {} } uri = /instances/953e7a82-4116-4de5-be0d-86445eadf394
6322023-12-12T05:49:47.851ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:33705 req_id = 2321550b-90d4-45c8-b2ef-669d24eeb708 resource = Database result = Ok(()) uri = /instances/953e7a82-4116-4de5-be0d-86445eadf394
6332023-12-12T05:49:47.852ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:33705 req_id = 2321550b-90d4-45c8-b2ef-669d24eeb708 roles = RoleSet { roles: {} } uri = /instances/953e7a82-4116-4de5-be0d-86445eadf394
6342023-12-12T05:49:47.852ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:33705 req_id = 2321550b-90d4-45c8-b2ef-669d24eeb708 resource = Database result = Ok(()) uri = /instances/953e7a82-4116-4de5-be0d-86445eadf394
6352023-12-12T05:49:47.853ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:33705 req_id = 2321550b-90d4-45c8-b2ef-669d24eeb708 roles = RoleSet { roles: {} } uri = /instances/953e7a82-4116-4de5-be0d-86445eadf394
6362023-12-12T05:49:47.854ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:33705 req_id = 2321550b-90d4-45c8-b2ef-669d24eeb708 resource = Database result = Ok(()) uri = /instances/953e7a82-4116-4de5-be0d-86445eadf394
6372023-12-12T05:49:47.855ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:33705 req_id = 2321550b-90d4-45c8-b2ef-669d24eeb708 roles = RoleSet { roles: {} } uri = /instances/953e7a82-4116-4de5-be0d-86445eadf394
6382023-12-12T05:49:47.855ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:33705 req_id = 2321550b-90d4-45c8-b2ef-669d24eeb708 resource = Database result = Ok(()) uri = /instances/953e7a82-4116-4de5-be0d-86445eadf394
6392023-12-12T05:49:47.856ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:33705 req_id = 2321550b-90d4-45c8-b2ef-669d24eeb708 roles = RoleSet { roles: {} } uri = /instances/953e7a82-4116-4de5-be0d-86445eadf394
6402023-12-12T05:49:47.856ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:33705 req_id = 2321550b-90d4-45c8-b2ef-669d24eeb708 resource = Database result = Ok(()) uri = /instances/953e7a82-4116-4de5-be0d-86445eadf394
6412023-12-12T05:49:47.857ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:33705 req_id = 2321550b-90d4-45c8-b2ef-669d24eeb708 roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } uri = /instances/953e7a82-4116-4de5-be0d-86445eadf394
6422023-12-12T05:49:47.860ZDEBGnexus (dropshot_internal): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:33705 req_id = 2321550b-90d4-45c8-b2ef-669d24eeb708 resource = Instance { parent: Project { parent: Silo { parent: Fleet, key: 3b6a6fc1-cb1d-4a52-8072-dd8e5d15f389, lookup_type: ById(3b6a6fc1-cb1d-4a52-8072-dd8e5d15f389) }, key: d5ed4b91-81cd-476b-8f8e-359627fd51d1, lookup_type: ById(d5ed4b91-81cd-476b-8f8e-359627fd51d1) }, key: 953e7a82-4116-4de5-be0d-86445eadf394, lookup_type: ById(953e7a82-4116-4de5-be0d-86445eadf394) } result = Ok(()) uri = /instances/953e7a82-4116-4de5-be0d-86445eadf394
6432023-12-12T05:49:47.860ZDEBGnexus: instance state generation already advanced, won't touch network config instance_id = 953e7a82-4116-4de5-be0d-86445eadf394
6442023-12-12T05:49:47.896ZINFOnexus: instance and vmm updated by sled agent file = nexus/src/app/instance.rs:1451 instance_id = 953e7a82-4116-4de5-be0d-86445eadf394 instance_updated = false propolis_id = 92388aab-2213-4c6b-8308-7fa9d8730c7c vmm_updated = true
6452023-12-12T05:49:47.896ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 46575 local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:33705 req_id = 2321550b-90d4-45c8-b2ef-669d24eeb708 response_code = 204 uri = /instances/953e7a82-4116-4de5-be0d-86445eadf394
6462023-12-12T05:49:52.284ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 508 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:36542 req_id = 3aff8f8e-92a0-4073-be2b-6fea3c6e6803 response_code = 200 uri = /metrics/collect/47e44ab6-ee0b-4407-95b8-c3af199399c8
6472023-12-12T05:50:02.040ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 12 reason = Timeout
6482023-12-12T05:50:02.040ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 10 reason = Timeout
6492023-12-12T05:50:02.040ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
6502023-12-12T05:50:02.041ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 10 reason = Timeout
6512023-12-12T05:50:02.041ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
6522023-12-12T05:50:02.041ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_external dns_group = external roles = RoleSet { roles: {} }
6532023-12-12T05:50:02.041ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 10 reason = Timeout
6542023-12-12T05:50:02.041ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 10 reason = Timeout
6552023-12-12T05:50:02.041ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_internal dns_group = internal roles = RoleSet { roles: {} }
6562023-12-12T05:50:02.041ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
6572023-12-12T05:50:02.041ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 10 reason = Timeout
6582023-12-12T05:50:02.041ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
6592023-12-12T05:50:02.041ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 12 reason = Timeout
6602023-12-12T05:50:02.041ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
6612023-12-12T05:50:02.041ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
6622023-12-12T05:50:02.041ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
6632023-12-12T05:50:02.041ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_external dns_group = external resource = Database result = Ok(())
6642023-12-12T05:50:02.041ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
6652023-12-12T05:50:02.041ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_internal dns_group = internal resource = Database result = Ok(())
6662023-12-12T05:50:02.042ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
6672023-12-12T05:50:02.042ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
6682023-12-12T05:50:02.042ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
6692023-12-12T05:50:02.042ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
6702023-12-12T05:50:02.043ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 16 reason = Timeout
6712023-12-12T05:50:02.043ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 16 reason = Timeout
6722023-12-12T05:50:02.043ZWARNnexus (BackgroundTasks): phantom disk task started background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:50
6732023-12-12T05:50:02.043ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
6742023-12-12T05:50:02.043ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
6752023-12-12T05:50:02.043ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6762023-12-12T05:50:02.043ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6772023-12-12T05:50:02.043ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6782023-12-12T05:50:02.044ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6792023-12-12T05:50:02.044ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6802023-12-12T05:50:02.044ZDEBGnexus (DpdClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
6812023-12-12T05:50:02.045ZDEBGnexus (DpdClient): client response result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "18f0959c-01bb-4235-8c08-8fa58cbfdbc9", "content-length": "1", "date": "Tue, 12 Dec 2023 05:50:01 GMT"} })
6822023-12-12T05:50:02.045ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
6832023-12-12T05:50:02.045ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
6842023-12-12T05:50:02.046ZWARNnexus (BackgroundTasks): phantom disk task done background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:96
6852023-12-12T05:50:02.046ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.243478ms iteration = 16
6862023-12-12T05:50:02.046ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 3.539592ms iteration = 16
6872023-12-12T05:50:02.047ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
6882023-12-12T05:50:02.047ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
6892023-12-12T05:50:02.047ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
6902023-12-12T05:50:02.047ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_internal dns_group = internal resource = Fleet result = Ok(())
6912023-12-12T05:50:02.047ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_internal dns_group = internal roles = RoleSet { roles: {} }
6922023-12-12T05:50:02.047ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_external dns_group = external resource = Fleet result = Ok(())
6932023-12-12T05:50:02.047ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_external dns_group = external roles = RoleSet { roles: {} }
6942023-12-12T05:50:02.048ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_internal dns_group = internal resource = Database result = Ok(())
6952023-12-12T05:50:02.048ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_external dns_group = external resource = Database result = Ok(())
6962023-12-12T05:50:02.049ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
6972023-12-12T05:50:02.049ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
6982023-12-12T05:50:02.049ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
6992023-12-12T05:50:02.049ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } dns_group = external
7002023-12-12T05:50:02.049ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 8.733573ms iteration = 10
7012023-12-12T05:50:02.049ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } dns_group = internal
7022023-12-12T05:50:02.049ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 8.711082ms iteration = 10
7032023-12-12T05:50:02.049ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
7042023-12-12T05:50:02.050ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
7052023-12-12T05:50:02.050ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
7062023-12-12T05:50:02.050ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
7072023-12-12T05:50:02.050ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
7082023-12-12T05:50:02.050ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
7092023-12-12T05:50:02.050ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7102023-12-12T05:50:02.051ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
7112023-12-12T05:50:02.051ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
7122023-12-12T05:50:02.051ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
7132023-12-12T05:50:02.051ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
7142023-12-12T05:50:02.051ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
7152023-12-12T05:50:02.051ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7162023-12-12T05:50:02.052ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7172023-12-12T05:50:02.055ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
7182023-12-12T05:50:02.055ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
7192023-12-12T05:50:02.055ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
7202023-12-12T05:50:02.055ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
7212023-12-12T05:50:02.055ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
7222023-12-12T05:50:02.056ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
7232023-12-12T05:50:02.056ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
7242023-12-12T05:50:02.056ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
7252023-12-12T05:50:02.057ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
7262023-12-12T05:50:02.057ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
7272023-12-12T05:50:02.057ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
7282023-12-12T05:50:02.057ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = aa66fdf1-2426-44d7-a757-e029895906aa dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
7292023-12-12T05:50:02.057ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
7302023-12-12T05:50:02.057ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
7312023-12-12T05:50:02.057ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
7322023-12-12T05:50:02.057ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7332023-12-12T05:50:02.058ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
7342023-12-12T05:50:02.058ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 54229ede-970c-45c1-9159-c834119f9f05 dns_zone_name = oxide.test found_so_far = 0 version = 2
7352023-12-12T05:50:02.058ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
7362023-12-12T05:50:02.058ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
7372023-12-12T05:50:02.058ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7382023-12-12T05:50:02.059ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
7392023-12-12T05:50:02.059ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
7402023-12-12T05:50:02.059ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
7412023-12-12T05:50:02.059ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7422023-12-12T05:50:02.061ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
7432023-12-12T05:50:02.061ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
7442023-12-12T05:50:02.061ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
7452023-12-12T05:50:02.063ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = aa66fdf1-2426-44d7-a757-e029895906aa dns_zone_name = control-plane.oxide.internal found_so_far = 51 version = 1
7462023-12-12T05:50:02.063ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
7472023-12-12T05:50:02.063ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2023-12-12 05:43:20.140927 UTC dns_group = internal generation = 1
7482023-12-12T05:50:02.063ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 22.420514ms iteration = 10
7492023-12-12T05:50:02.064ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
7502023-12-12T05:50:02.064ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
7512023-12-12T05:50:02.064ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
7522023-12-12T05:50:02.065ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 54229ede-970c-45c1-9159-c834119f9f05 dns_zone_name = oxide.test found_so_far = 1 version = 2
7532023-12-12T05:50:02.065ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
7542023-12-12T05:50:02.065ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2023-12-12 05:43:20.566607 UTC dns_group = external generation = 2
7552023-12-12T05:50:02.065ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 24.437437ms iteration = 10
7562023-12-12T05:50:02.066ZINFOnexus (BackgroundTasks): found Silo/DNS/TLS config (changed) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2023-12-12T05:43:02.185336Z, time_modified: 2023-12-12T05:43:02.185336Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 3b6a6fc1-cb1d-4a52-8072-dd8e5d15f389, db_silo: Silo { identity: SiloIdentity { id: 3b6a6fc1-cb1d-4a52-8072-dd8e5d15f389, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2023-12-12T05:43:20.510842Z, time_modified: 2023-12-12T05:43:20.510842Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "09cd4777b7e66ecc555780ddf2e4364cfefa9836047f5f5d63dfec10c20bcbfe" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 3b6a6fc1-cb1d-4a52-8072-dd8e5d15f389, db_silo: Silo { identity: SiloIdentity { id: 3b6a6fc1-cb1d-4a52-8072-dd8e5d15f389, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2023-12-12T05:43:20.510842Z, time_modified: 2023-12-12T05:43:20.510842Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "09cd4777b7e66ecc555780ddf2e4364cfefa9836047f5f5d63dfec10c20bcbfe" }] }) } file = nexus/src/app/background/external_endpoints.rs:105
7572023-12-12T05:50:02.066ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 25.882955ms iteration = 10
7582023-12-12T05:50:02.174ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
7592023-12-12T05:50:02.176ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
7602023-12-12T05:50:02.284ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 516 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:36542 req_id = 7616a919-89f0-4906-9cfb-a0c5e055df11 response_code = 200 uri = /metrics/collect/47e44ab6-ee0b-4407-95b8-c3af199399c8
7612023-12-12T05:50:02.301ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
7622023-12-12T05:50:02.301ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::8)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "f66ed496-5b37-4d5a-85ae-b9f683c5f9bb", "date": "Tue, 12 Dec 2023 05:50:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
7632023-12-12T05:50:02.301ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/dns_propagation.rs:173 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
7642023-12-12T05:50:02.303ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
7652023-12-12T05:50:02.303ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::9)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "b36d2a04-0aba-4922-a7ef-0d8c823f0c87", "date": "Tue, 12 Dec 2023 05:50:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
7662023-12-12T05:50:02.303ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/dns_propagation.rs:173 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
7672023-12-12T05:50:02.304ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/dns_propagation.rs:110 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
7682023-12-12T05:50:02.304ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 263.447478ms iteration = 12
7692023-12-12T05:50:02.334ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
7702023-12-12T05:50:02.335ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:1::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "d2a54517-05e3-49ef-ba1f-69659cd39404", "date": "Tue, 12 Dec 2023 05:50:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
7712023-12-12T05:50:02.335ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/dns_propagation.rs:173 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
7722023-12-12T05:50:02.336ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:2::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "e25ac159-615a-477e-9d63-3cfeb6d66ecf", "date": "Tue, 12 Dec 2023 05:50:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
7732023-12-12T05:50:02.336ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/dns_propagation.rs:173 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
7742023-12-12T05:50:02.338ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:3::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "4cfaf7db-9150-44b8-b3af-f6c825f044d0", "date": "Tue, 12 Dec 2023 05:50:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
7752023-12-12T05:50:02.338ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/dns_propagation.rs:173 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
7762023-12-12T05:50:02.340ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/dns_propagation.rs:110 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
7772023-12-12T05:50:02.340ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 299.574853ms iteration = 12
7782023-12-12T05:50:04.892ZINFOnexus: received new runtime state from sled agent file = nexus/src/app/instance.rs:1315 instance_id = 953e7a82-4116-4de5-be0d-86445eadf394 instance_state = InstanceRuntimeState { propolis_id: Some(92388aab-2213-4c6b-8308-7fa9d8730c7c), dst_propolis_id: None, migration_id: None, gen: Generation(3), time_updated: 2023-12-12T05:49:39.966360524Z } propolis_id = 92388aab-2213-4c6b-8308-7fa9d8730c7c vmm_state = VmmRuntimeState { state: Stopping, gen: Generation(5), time_updated: 2023-12-12T05:50:04.892044756Z }
7792023-12-12T05:50:04.892ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:33705 req_id = c2434e6b-05f5-4146-947b-10b53cee8989 roles = RoleSet { roles: {} } uri = /instances/953e7a82-4116-4de5-be0d-86445eadf394
7802023-12-12T05:50:04.892ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:33705 req_id = c2434e6b-05f5-4146-947b-10b53cee8989 resource = Database result = Ok(()) uri = /instances/953e7a82-4116-4de5-be0d-86445eadf394
7812023-12-12T05:50:04.894ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:33705 req_id = c2434e6b-05f5-4146-947b-10b53cee8989 roles = RoleSet { roles: {} } uri = /instances/953e7a82-4116-4de5-be0d-86445eadf394
7822023-12-12T05:50:04.894ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:33705 req_id = c2434e6b-05f5-4146-947b-10b53cee8989 resource = Database result = Ok(()) uri = /instances/953e7a82-4116-4de5-be0d-86445eadf394
7832023-12-12T05:50:04.895ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:33705 req_id = c2434e6b-05f5-4146-947b-10b53cee8989 roles = RoleSet { roles: {} } uri = /instances/953e7a82-4116-4de5-be0d-86445eadf394
7842023-12-12T05:50:04.895ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:33705 req_id = c2434e6b-05f5-4146-947b-10b53cee8989 resource = Database result = Ok(()) uri = /instances/953e7a82-4116-4de5-be0d-86445eadf394
7852023-12-12T05:50:04.896ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:33705 req_id = c2434e6b-05f5-4146-947b-10b53cee8989 roles = RoleSet { roles: {} } uri = /instances/953e7a82-4116-4de5-be0d-86445eadf394
7862023-12-12T05:50:04.896ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:33705 req_id = c2434e6b-05f5-4146-947b-10b53cee8989 resource = Database result = Ok(()) uri = /instances/953e7a82-4116-4de5-be0d-86445eadf394
7872023-12-12T05:50:04.897ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:33705 req_id = c2434e6b-05f5-4146-947b-10b53cee8989 roles = RoleSet { roles: {} } uri = /instances/953e7a82-4116-4de5-be0d-86445eadf394
7882023-12-12T05:50:04.897ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:33705 req_id = c2434e6b-05f5-4146-947b-10b53cee8989 resource = Database result = Ok(()) uri = /instances/953e7a82-4116-4de5-be0d-86445eadf394
7892023-12-12T05:50:04.898ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:33705 req_id = c2434e6b-05f5-4146-947b-10b53cee8989 roles = RoleSet { roles: {} } uri = /instances/953e7a82-4116-4de5-be0d-86445eadf394
7902023-12-12T05:50:04.899ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:33705 req_id = c2434e6b-05f5-4146-947b-10b53cee8989 resource = Database result = Ok(()) uri = /instances/953e7a82-4116-4de5-be0d-86445eadf394
7912023-12-12T05:50:04.900ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:33705 req_id = c2434e6b-05f5-4146-947b-10b53cee8989 roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } uri = /instances/953e7a82-4116-4de5-be0d-86445eadf394
7922023-12-12T05:50:04.903ZDEBGnexus (dropshot_internal): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:33705 req_id = c2434e6b-05f5-4146-947b-10b53cee8989 resource = Instance { parent: Project { parent: Silo { parent: Fleet, key: 3b6a6fc1-cb1d-4a52-8072-dd8e5d15f389, lookup_type: ById(3b6a6fc1-cb1d-4a52-8072-dd8e5d15f389) }, key: d5ed4b91-81cd-476b-8f8e-359627fd51d1, lookup_type: ById(d5ed4b91-81cd-476b-8f8e-359627fd51d1) }, key: 953e7a82-4116-4de5-be0d-86445eadf394, lookup_type: ById(953e7a82-4116-4de5-be0d-86445eadf394) } result = Ok(()) uri = /instances/953e7a82-4116-4de5-be0d-86445eadf394
7932023-12-12T05:50:04.903ZDEBGnexus: instance state generation already advanced, won't touch network config instance_id = 953e7a82-4116-4de5-be0d-86445eadf394
7942023-12-12T05:50:04.947ZINFOnexus: instance and vmm updated by sled agent file = nexus/src/app/instance.rs:1451 instance_id = 953e7a82-4116-4de5-be0d-86445eadf394 instance_updated = false propolis_id = 92388aab-2213-4c6b-8308-7fa9d8730c7c vmm_updated = true
7952023-12-12T05:50:04.947ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 54876 local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:33705 req_id = c2434e6b-05f5-4146-947b-10b53cee8989 response_code = 204 uri = /instances/953e7a82-4116-4de5-be0d-86445eadf394
7962023-12-12T05:50:04.948ZINFOnexus: received new runtime state from sled agent file = nexus/src/app/instance.rs:1315 instance_id = 953e7a82-4116-4de5-be0d-86445eadf394 instance_state = InstanceRuntimeState { propolis_id: Some(92388aab-2213-4c6b-8308-7fa9d8730c7c), dst_propolis_id: None, migration_id: None, gen: Generation(3), time_updated: 2023-12-12T05:49:39.966360524Z } propolis_id = 92388aab-2213-4c6b-8308-7fa9d8730c7c vmm_state = VmmRuntimeState { state: Stopping, gen: Generation(6), time_updated: 2023-12-12T05:50:04.947350436Z }
7972023-12-12T05:50:04.948ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:33705 req_id = f79b2588-939f-46f8-907d-6ed57fde3ab9 roles = RoleSet { roles: {} } uri = /instances/953e7a82-4116-4de5-be0d-86445eadf394
7982023-12-12T05:50:04.948ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:33705 req_id = f79b2588-939f-46f8-907d-6ed57fde3ab9 resource = Database result = Ok(()) uri = /instances/953e7a82-4116-4de5-be0d-86445eadf394
7992023-12-12T05:50:04.949ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:33705 req_id = f79b2588-939f-46f8-907d-6ed57fde3ab9 roles = RoleSet { roles: {} } uri = /instances/953e7a82-4116-4de5-be0d-86445eadf394
8002023-12-12T05:50:04.950ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:33705 req_id = f79b2588-939f-46f8-907d-6ed57fde3ab9 resource = Database result = Ok(()) uri = /instances/953e7a82-4116-4de5-be0d-86445eadf394
8012023-12-12T05:50:04.951ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:33705 req_id = f79b2588-939f-46f8-907d-6ed57fde3ab9 roles = RoleSet { roles: {} } uri = /instances/953e7a82-4116-4de5-be0d-86445eadf394
8022023-12-12T05:50:04.951ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:33705 req_id = f79b2588-939f-46f8-907d-6ed57fde3ab9 resource = Database result = Ok(()) uri = /instances/953e7a82-4116-4de5-be0d-86445eadf394
8032023-12-12T05:50:04.952ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:33705 req_id = f79b2588-939f-46f8-907d-6ed57fde3ab9 roles = RoleSet { roles: {} } uri = /instances/953e7a82-4116-4de5-be0d-86445eadf394
8042023-12-12T05:50:04.952ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:33705 req_id = f79b2588-939f-46f8-907d-6ed57fde3ab9 resource = Database result = Ok(()) uri = /instances/953e7a82-4116-4de5-be0d-86445eadf394
8052023-12-12T05:50:04.953ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:33705 req_id = f79b2588-939f-46f8-907d-6ed57fde3ab9 roles = RoleSet { roles: {} } uri = /instances/953e7a82-4116-4de5-be0d-86445eadf394
8062023-12-12T05:50:04.954ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:33705 req_id = f79b2588-939f-46f8-907d-6ed57fde3ab9 resource = Database result = Ok(()) uri = /instances/953e7a82-4116-4de5-be0d-86445eadf394
8072023-12-12T05:50:04.955ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:33705 req_id = f79b2588-939f-46f8-907d-6ed57fde3ab9 roles = RoleSet { roles: {} } uri = /instances/953e7a82-4116-4de5-be0d-86445eadf394
8082023-12-12T05:50:04.955ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:33705 req_id = f79b2588-939f-46f8-907d-6ed57fde3ab9 resource = Database result = Ok(()) uri = /instances/953e7a82-4116-4de5-be0d-86445eadf394
8092023-12-12T05:50:04.956ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:33705 req_id = f79b2588-939f-46f8-907d-6ed57fde3ab9 roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } uri = /instances/953e7a82-4116-4de5-be0d-86445eadf394
8102023-12-12T05:50:04.959ZDEBGnexus (dropshot_internal): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:33705 req_id = f79b2588-939f-46f8-907d-6ed57fde3ab9 resource = Instance { parent: Project { parent: Silo { parent: Fleet, key: 3b6a6fc1-cb1d-4a52-8072-dd8e5d15f389, lookup_type: ById(3b6a6fc1-cb1d-4a52-8072-dd8e5d15f389) }, key: d5ed4b91-81cd-476b-8f8e-359627fd51d1, lookup_type: ById(d5ed4b91-81cd-476b-8f8e-359627fd51d1) }, key: 953e7a82-4116-4de5-be0d-86445eadf394, lookup_type: ById(953e7a82-4116-4de5-be0d-86445eadf394) } result = Ok(()) uri = /instances/953e7a82-4116-4de5-be0d-86445eadf394
8112023-12-12T05:50:04.959ZDEBGnexus: instance state generation already advanced, won't touch network config instance_id = 953e7a82-4116-4de5-be0d-86445eadf394
8122023-12-12T05:50:04.966ZINFOnexus: instance and vmm updated by sled agent file = nexus/src/app/instance.rs:1451 instance_id = 953e7a82-4116-4de5-be0d-86445eadf394 instance_updated = false propolis_id = 92388aab-2213-4c6b-8308-7fa9d8730c7c vmm_updated = true
8132023-12-12T05:50:04.966ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 18778 local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:33705 req_id = f79b2588-939f-46f8-907d-6ed57fde3ab9 response_code = 204 uri = /instances/953e7a82-4116-4de5-be0d-86445eadf394
8142023-12-12T05:50:07.295ZINFOnexus: received new runtime state from sled agent file = nexus/src/app/instance.rs:1315 instance_id = 953e7a82-4116-4de5-be0d-86445eadf394 instance_state = InstanceRuntimeState { propolis_id: None, dst_propolis_id: None, migration_id: None, gen: Generation(5), time_updated: 2023-12-12T05:50:05.082207843Z } propolis_id = 92388aab-2213-4c6b-8308-7fa9d8730c7c vmm_state = VmmRuntimeState { state: Destroyed, gen: Generation(7), time_updated: 2023-12-12T05:50:05.082207843Z }
8152023-12-12T05:50:07.295ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:33705 req_id = 0849120b-e717-4992-9dd6-5d7c50beb0ab roles = RoleSet { roles: {} } uri = /instances/953e7a82-4116-4de5-be0d-86445eadf394
8162023-12-12T05:50:07.295ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:33705 req_id = 0849120b-e717-4992-9dd6-5d7c50beb0ab resource = Database result = Ok(()) uri = /instances/953e7a82-4116-4de5-be0d-86445eadf394
8172023-12-12T05:50:07.298ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:33705 req_id = 0849120b-e717-4992-9dd6-5d7c50beb0ab roles = RoleSet { roles: {} } uri = /instances/953e7a82-4116-4de5-be0d-86445eadf394
8182023-12-12T05:50:07.298ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:33705 req_id = 0849120b-e717-4992-9dd6-5d7c50beb0ab resource = Database result = Ok(()) uri = /instances/953e7a82-4116-4de5-be0d-86445eadf394
8192023-12-12T05:50:07.299ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:33705 req_id = 0849120b-e717-4992-9dd6-5d7c50beb0ab roles = RoleSet { roles: {} } uri = /instances/953e7a82-4116-4de5-be0d-86445eadf394
8202023-12-12T05:50:07.299ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:33705 req_id = 0849120b-e717-4992-9dd6-5d7c50beb0ab resource = Database result = Ok(()) uri = /instances/953e7a82-4116-4de5-be0d-86445eadf394
8212023-12-12T05:50:07.301ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:33705 req_id = 0849120b-e717-4992-9dd6-5d7c50beb0ab roles = RoleSet { roles: {} } uri = /instances/953e7a82-4116-4de5-be0d-86445eadf394
8222023-12-12T05:50:07.301ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:33705 req_id = 0849120b-e717-4992-9dd6-5d7c50beb0ab resource = Database result = Ok(()) uri = /instances/953e7a82-4116-4de5-be0d-86445eadf394
8232023-12-12T05:50:07.302ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:33705 req_id = 0849120b-e717-4992-9dd6-5d7c50beb0ab roles = RoleSet { roles: {} } uri = /instances/953e7a82-4116-4de5-be0d-86445eadf394
8242023-12-12T05:50:07.302ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:33705 req_id = 0849120b-e717-4992-9dd6-5d7c50beb0ab resource = Database result = Ok(()) uri = /instances/953e7a82-4116-4de5-be0d-86445eadf394
8252023-12-12T05:50:07.304ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:33705 req_id = 0849120b-e717-4992-9dd6-5d7c50beb0ab roles = RoleSet { roles: {} } uri = /instances/953e7a82-4116-4de5-be0d-86445eadf394
8262023-12-12T05:50:07.304ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:33705 req_id = 0849120b-e717-4992-9dd6-5d7c50beb0ab resource = Database result = Ok(()) uri = /instances/953e7a82-4116-4de5-be0d-86445eadf394
8272023-12-12T05:50:07.305ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:33705 req_id = 0849120b-e717-4992-9dd6-5d7c50beb0ab roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } uri = /instances/953e7a82-4116-4de5-be0d-86445eadf394
8282023-12-12T05:50:07.308ZDEBGnexus (dropshot_internal): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:33705 req_id = 0849120b-e717-4992-9dd6-5d7c50beb0ab resource = Instance { parent: Project { parent: Silo { parent: Fleet, key: 3b6a6fc1-cb1d-4a52-8072-dd8e5d15f389, lookup_type: ById(3b6a6fc1-cb1d-4a52-8072-dd8e5d15f389) }, key: d5ed4b91-81cd-476b-8f8e-359627fd51d1, lookup_type: ById(d5ed4b91-81cd-476b-8f8e-359627fd51d1) }, key: 953e7a82-4116-4de5-be0d-86445eadf394, lookup_type: ById(953e7a82-4116-4de5-be0d-86445eadf394) } result = Ok(()) uri = /instances/953e7a82-4116-4de5-be0d-86445eadf394
8292023-12-12T05:50:07.308ZINFOnexus: instance cleared its Propolis ID, cleaning network config file = nexus/src/app/instance_network.rs:626 instance_id = 953e7a82-4116-4de5-be0d-86445eadf394 propolis_id = Some(92388aab-2213-4c6b-8308-7fa9d8730c7c)
8302023-12-12T05:50:07.308ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:33705 req_id = 0849120b-e717-4992-9dd6-5d7c50beb0ab roles = RoleSet { roles: {} } uri = /instances/953e7a82-4116-4de5-be0d-86445eadf394
8312023-12-12T05:50:07.308ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:33705 req_id = 0849120b-e717-4992-9dd6-5d7c50beb0ab resource = Database result = Ok(()) uri = /instances/953e7a82-4116-4de5-be0d-86445eadf394
8322023-12-12T05:50:07.309ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:33705 req_id = 0849120b-e717-4992-9dd6-5d7c50beb0ab roles = RoleSet { roles: {} } uri = /instances/953e7a82-4116-4de5-be0d-86445eadf394
8332023-12-12T05:50:07.310ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:33705 req_id = 0849120b-e717-4992-9dd6-5d7c50beb0ab resource = Database result = Ok(()) uri = /instances/953e7a82-4116-4de5-be0d-86445eadf394
8342023-12-12T05:50:07.311ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:33705 req_id = 0849120b-e717-4992-9dd6-5d7c50beb0ab roles = RoleSet { roles: {} } uri = /instances/953e7a82-4116-4de5-be0d-86445eadf394
8352023-12-12T05:50:07.311ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:33705 req_id = 0849120b-e717-4992-9dd6-5d7c50beb0ab resource = Database result = Ok(()) uri = /instances/953e7a82-4116-4de5-be0d-86445eadf394
8362023-12-12T05:50:07.312ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:33705 req_id = 0849120b-e717-4992-9dd6-5d7c50beb0ab roles = RoleSet { roles: {} } uri = /instances/953e7a82-4116-4de5-be0d-86445eadf394
8372023-12-12T05:50:07.312ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:33705 req_id = 0849120b-e717-4992-9dd6-5d7c50beb0ab resource = Database result = Ok(()) uri = /instances/953e7a82-4116-4de5-be0d-86445eadf394
8382023-12-12T05:50:07.313ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:33705 req_id = 0849120b-e717-4992-9dd6-5d7c50beb0ab roles = RoleSet { roles: {} } uri = /instances/953e7a82-4116-4de5-be0d-86445eadf394
8392023-12-12T05:50:07.313ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:33705 req_id = 0849120b-e717-4992-9dd6-5d7c50beb0ab resource = Database result = Ok(()) uri = /instances/953e7a82-4116-4de5-be0d-86445eadf394
8402023-12-12T05:50:07.314ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:33705 req_id = 0849120b-e717-4992-9dd6-5d7c50beb0ab roles = RoleSet { roles: {} } uri = /instances/953e7a82-4116-4de5-be0d-86445eadf394
8412023-12-12T05:50:07.314ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:33705 req_id = 0849120b-e717-4992-9dd6-5d7c50beb0ab resource = Database result = Ok(()) uri = /instances/953e7a82-4116-4de5-be0d-86445eadf394
8422023-12-12T05:50:07.315ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:33705 req_id = 0849120b-e717-4992-9dd6-5d7c50beb0ab roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } uri = /instances/953e7a82-4116-4de5-be0d-86445eadf394
8432023-12-12T05:50:07.318ZDEBGnexus (dropshot_internal): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:33705 req_id = 0849120b-e717-4992-9dd6-5d7c50beb0ab resource = Instance { parent: Project { parent: Silo { parent: Fleet, key: 3b6a6fc1-cb1d-4a52-8072-dd8e5d15f389, lookup_type: ById(3b6a6fc1-cb1d-4a52-8072-dd8e5d15f389) }, key: d5ed4b91-81cd-476b-8f8e-359627fd51d1, lookup_type: ById(d5ed4b91-81cd-476b-8f8e-359627fd51d1) }, key: 953e7a82-4116-4de5-be0d-86445eadf394, lookup_type: ById(953e7a82-4116-4de5-be0d-86445eadf394) } result = Ok(()) uri = /instances/953e7a82-4116-4de5-be0d-86445eadf394
8442023-12-12T05:50:07.318ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:33705 req_id = 0849120b-e717-4992-9dd6-5d7c50beb0ab roles = RoleSet { roles: {} } uri = /instances/953e7a82-4116-4de5-be0d-86445eadf394
8452023-12-12T05:50:07.318ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:33705 req_id = 0849120b-e717-4992-9dd6-5d7c50beb0ab resource = Database result = Ok(()) uri = /instances/953e7a82-4116-4de5-be0d-86445eadf394
8462023-12-12T05:50:07.319ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:33705 req_id = 0849120b-e717-4992-9dd6-5d7c50beb0ab roles = RoleSet { roles: {} } uri = /instances/953e7a82-4116-4de5-be0d-86445eadf394
8472023-12-12T05:50:07.319ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:33705 req_id = 0849120b-e717-4992-9dd6-5d7c50beb0ab resource = Database result = Ok(()) uri = /instances/953e7a82-4116-4de5-be0d-86445eadf394
8482023-12-12T05:50:07.320ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:33705 req_id = 0849120b-e717-4992-9dd6-5d7c50beb0ab roles = RoleSet { roles: {} } uri = /instances/953e7a82-4116-4de5-be0d-86445eadf394
8492023-12-12T05:50:07.320ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:33705 req_id = 0849120b-e717-4992-9dd6-5d7c50beb0ab resource = Database result = Ok(()) uri = /instances/953e7a82-4116-4de5-be0d-86445eadf394
8502023-12-12T05:50:07.321ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:33705 req_id = 0849120b-e717-4992-9dd6-5d7c50beb0ab roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } uri = /instances/953e7a82-4116-4de5-be0d-86445eadf394
8512023-12-12T05:50:07.324ZDEBGnexus (dropshot_internal): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:33705 req_id = 0849120b-e717-4992-9dd6-5d7c50beb0ab resource = Instance { parent: Project { parent: Silo { parent: Fleet, key: 3b6a6fc1-cb1d-4a52-8072-dd8e5d15f389, lookup_type: ById(3b6a6fc1-cb1d-4a52-8072-dd8e5d15f389) }, key: d5ed4b91-81cd-476b-8f8e-359627fd51d1, lookup_type: ById(d5ed4b91-81cd-476b-8f8e-359627fd51d1) }, key: 953e7a82-4116-4de5-be0d-86445eadf394, lookup_type: ById(953e7a82-4116-4de5-be0d-86445eadf394) } result = Ok(()) uri = /instances/953e7a82-4116-4de5-be0d-86445eadf394
8522023-12-12T05:50:07.324ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:33705 req_id = 0849120b-e717-4992-9dd6-5d7c50beb0ab roles = RoleSet { roles: {} } uri = /instances/953e7a82-4116-4de5-be0d-86445eadf394
8532023-12-12T05:50:07.324ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:33705 req_id = 0849120b-e717-4992-9dd6-5d7c50beb0ab resource = Database result = Ok(()) uri = /instances/953e7a82-4116-4de5-be0d-86445eadf394
8542023-12-12T05:50:07.362ZDEBGnexus (InstanceAllocator): roles roles = RoleSet { roles: {} }
8552023-12-12T05:50:07.362ZDEBGnexus (InstanceAllocator): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Database result = Ok(())
8562023-12-12T05:50:07.363ZDEBGnexus (InstanceAllocator): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "viewer")} }
8572023-12-12T05:50:07.364ZDEBGnexus (InstanceAllocator): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Fleet result = Ok(())
8582023-12-12T05:50:07.364ZDEBGnexus (InstanceAllocator): roles roles = RoleSet { roles: {} }
8592023-12-12T05:50:07.364ZDEBGnexus (InstanceAllocator): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Database result = Ok(())
8602023-12-12T05:50:07.366ZDEBGnexus (InstanceAllocator): roles roles = RoleSet { roles: {} }
8612023-12-12T05:50:07.366ZDEBGnexus (InstanceAllocator): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Database result = Ok(())
8622023-12-12T05:50:07.368ZDEBGnexus (InstanceAllocator): roles roles = RoleSet { roles: {} }
8632023-12-12T05:50:07.368ZDEBGnexus (InstanceAllocator): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Database result = Ok(())
8642023-12-12T05:50:07.369ZDEBGnexus (InstanceAllocator): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "viewer")} }
8652023-12-12T05:50:07.371ZDEBGnexus (InstanceAllocator): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Sled { parent: Fleet, key: 6d688fb7-62a5-4d58-b41d-c16b04ef6f24, lookup_type: ById(6d688fb7-62a5-4d58-b41d-c16b04ef6f24) } result = Ok(())
8662023-12-12T05:50:07.401ZDEBGnexus: client request SledAgent = 6d688fb7-62a5-4d58-b41d-c16b04ef6f24 body = Some(Body) method = DELETE uri = http://[fd00:1122:3344:101::1]:12345/v2p/58a8d492-9a29-4aca-a2bd-fb170581c3fe
8672023-12-12T05:50:07.401ZDEBGnexus: client response SledAgent = 6d688fb7-62a5-4d58-b41d-c16b04ef6f24 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/v2p/58a8d492-9a29-4aca-a2bd-fb170581c3fe", query: None, fragment: None }, status: 204, headers: {"x-request-id": "d666cf7b-5895-40de-bd13-fbb457bea334", "date": "Tue, 12 Dec 2023 05:50:07 GMT"} })
8682023-12-12T05:50:07.402ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:33705 req_id = 0849120b-e717-4992-9dd6-5d7c50beb0ab roles = RoleSet { roles: {} } uri = /instances/953e7a82-4116-4de5-be0d-86445eadf394
8692023-12-12T05:50:07.402ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:33705 req_id = 0849120b-e717-4992-9dd6-5d7c50beb0ab resource = Database result = Ok(()) uri = /instances/953e7a82-4116-4de5-be0d-86445eadf394
8702023-12-12T05:50:07.406ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:33705 req_id = 0849120b-e717-4992-9dd6-5d7c50beb0ab roles = RoleSet { roles: {} } uri = /instances/953e7a82-4116-4de5-be0d-86445eadf394
8712023-12-12T05:50:07.407ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:33705 req_id = 0849120b-e717-4992-9dd6-5d7c50beb0ab resource = Database result = Ok(()) uri = /instances/953e7a82-4116-4de5-be0d-86445eadf394
8722023-12-12T05:50:07.408ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:33705 req_id = 0849120b-e717-4992-9dd6-5d7c50beb0ab roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } uri = /instances/953e7a82-4116-4de5-be0d-86445eadf394
8732023-12-12T05:50:07.409ZDEBGnexus (dropshot_internal): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:33705 req_id = 0849120b-e717-4992-9dd6-5d7c50beb0ab resource = Fleet result = Ok(()) uri = /instances/953e7a82-4116-4de5-be0d-86445eadf394
8742023-12-12T05:50:07.409ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:33705 req_id = 0849120b-e717-4992-9dd6-5d7c50beb0ab roles = RoleSet { roles: {} } uri = /instances/953e7a82-4116-4de5-be0d-86445eadf394
8752023-12-12T05:50:07.410ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:33705 req_id = 0849120b-e717-4992-9dd6-5d7c50beb0ab resource = Database result = Ok(()) uri = /instances/953e7a82-4116-4de5-be0d-86445eadf394
8762023-12-12T05:50:07.415ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:33705 req_id = 0849120b-e717-4992-9dd6-5d7c50beb0ab roles = RoleSet { roles: {} } uri = /instances/953e7a82-4116-4de5-be0d-86445eadf394
8772023-12-12T05:50:07.415ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:33705 req_id = 0849120b-e717-4992-9dd6-5d7c50beb0ab resource = Database result = Ok(()) uri = /instances/953e7a82-4116-4de5-be0d-86445eadf394
8782023-12-12T05:50:07.419ZWARNnexus: no matching nat entries to soft delete file = nexus/src/app/instance_network.rs:543
8792023-12-12T05:50:07.419ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:33705 req_id = 0849120b-e717-4992-9dd6-5d7c50beb0ab roles = RoleSet { roles: {} } uri = /instances/953e7a82-4116-4de5-be0d-86445eadf394
8802023-12-12T05:50:07.419ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:33705 req_id = 0849120b-e717-4992-9dd6-5d7c50beb0ab resource = Database result = Ok(()) uri = /instances/953e7a82-4116-4de5-be0d-86445eadf394
8812023-12-12T05:50:07.421ZWARNnexus: no matching nat entries to soft delete file = nexus/src/app/instance_network.rs:543
8822023-12-12T05:50:07.421ZDEBGnexus: notifying dendrite of updates instance_id = 953e7a82-4116-4de5-be0d-86445eadf394 switch = switch0
8832023-12-12T05:50:07.421ZDEBGnexus (DpdClient): client request body = None method = POST uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/trigger
8842023-12-12T05:50:07.421ZDEBGnexus (DpdClient): client response result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/trigger", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "32d3eafc-fbf8-4132-8273-60a6fe8519ee", "content-length": "4", "date": "Tue, 12 Dec 2023 05:50:07 GMT"} })
8852023-12-12T05:50:07.422ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:33705 req_id = 0849120b-e717-4992-9dd6-5d7c50beb0ab roles = RoleSet { roles: {} } uri = /instances/953e7a82-4116-4de5-be0d-86445eadf394
8862023-12-12T05:50:07.422ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:33705 req_id = 0849120b-e717-4992-9dd6-5d7c50beb0ab resource = Database result = Ok(()) uri = /instances/953e7a82-4116-4de5-be0d-86445eadf394
8872023-12-12T05:50:07.434ZDEBGnexus: deleted metric producer assignment collector_id = 8c5057a0-d3bf-418d-9fa5-e6dc3ef919a3 producer_id = 953e7a82-4116-4de5-be0d-86445eadf394
8882023-12-12T05:50:07.466ZINFOnexus: registered oximeter collector client file = nexus/src/app/oximeter.rs:374 id = 953e7a82-4116-4de5-be0d-86445eadf394
8892023-12-12T05:50:07.466ZDEBGnexus: client request body = None method = DELETE oximeter-collector = 953e7a82-4116-4de5-be0d-86445eadf394 uri = http://[fd00:1122:3344:101::d]:12223/producers/953e7a82-4116-4de5-be0d-86445eadf394
8902023-12-12T05:50:07.467ZDEBGnexus: client response oximeter-collector = 953e7a82-4116-4de5-be0d-86445eadf394 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::d)), port: Some(12223), path: "/producers/953e7a82-4116-4de5-be0d-86445eadf394", query: None, fragment: None }, status: 204, headers: {"x-request-id": "d3dabd75-410a-401f-9a9b-c9827f0000a2", "date": "Tue, 12 Dec 2023 05:50:07 GMT"} })
8912023-12-12T05:50:07.467ZDEBGnexus: successfully deleted producer from collector address = [fd00:1122:3344:101::d]:12223 collector_id = 8c5057a0-d3bf-418d-9fa5-e6dc3ef919a3 producer_id = 953e7a82-4116-4de5-be0d-86445eadf394
8922023-12-12T05:50:07.478ZINFOnexus: vmm is terminated, cleaning up resources file = nexus/src/app/instance.rs:1428 instance_id = 953e7a82-4116-4de5-be0d-86445eadf394 propolis_id = 92388aab-2213-4c6b-8308-7fa9d8730c7c
8932023-12-12T05:50:07.478ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:33705 req_id = 0849120b-e717-4992-9dd6-5d7c50beb0ab roles = RoleSet { roles: {} } uri = /instances/953e7a82-4116-4de5-be0d-86445eadf394
8942023-12-12T05:50:07.479ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:33705 req_id = 0849120b-e717-4992-9dd6-5d7c50beb0ab resource = Database result = Ok(()) uri = /instances/953e7a82-4116-4de5-be0d-86445eadf394
8952023-12-12T05:50:07.482ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:33705 req_id = 0849120b-e717-4992-9dd6-5d7c50beb0ab roles = RoleSet { roles: {} } uri = /instances/953e7a82-4116-4de5-be0d-86445eadf394
8962023-12-12T05:50:07.482ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:33705 req_id = 0849120b-e717-4992-9dd6-5d7c50beb0ab resource = Database result = Ok(()) uri = /instances/953e7a82-4116-4de5-be0d-86445eadf394
8972023-12-12T05:50:07.515ZINFOnexus: instance and vmm updated by sled agent file = nexus/src/app/instance.rs:1451 instance_id = 953e7a82-4116-4de5-be0d-86445eadf394 instance_updated = true propolis_id = 92388aab-2213-4c6b-8308-7fa9d8730c7c vmm_updated = true
8982023-12-12T05:50:07.515ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 220037 local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:33705 req_id = 0849120b-e717-4992-9dd6-5d7c50beb0ab response_code = 204 uri = /instances/953e7a82-4116-4de5-be0d-86445eadf394
8992023-12-12T05:50:12.284ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 856 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:36542 req_id = e3df8202-5f86-4d61-983b-793f56cda408 response_code = 200 uri = /metrics/collect/47e44ab6-ee0b-4407-95b8-c3af199399c8
9002023-12-12T05:50:22.285ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 799 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:36542 req_id = 96af566c-5875-4787-afe2-828e7a4e62c8 response_code = 200 uri = /metrics/collect/47e44ab6-ee0b-4407-95b8-c3af199399c8
9012023-12-12T05:50:32.043ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 17 reason = Timeout
9022023-12-12T05:50:32.043ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 17 reason = Timeout
9032023-12-12T05:50:32.043ZWARNnexus (BackgroundTasks): phantom disk task started background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:50
9042023-12-12T05:50:32.043ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
9052023-12-12T05:50:32.044ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
9062023-12-12T05:50:32.045ZDEBGnexus (DpdClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
9072023-12-12T05:50:32.045ZDEBGnexus (DpdClient): client response result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "deb43e19-589a-4f61-a39d-492dee0e3714", "content-length": "1", "date": "Tue, 12 Dec 2023 05:50:31 GMT"} })
9082023-12-12T05:50:32.045ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
9092023-12-12T05:50:32.046ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
9102023-12-12T05:50:32.046ZWARNnexus (BackgroundTasks): phantom disk task done background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:96
9112023-12-12T05:50:32.046ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 2.778143ms iteration = 17
9122023-12-12T05:50:32.047ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 3.400669ms iteration = 17
9132023-12-12T05:50:32.284ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 430 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:36542 req_id = fea0937b-da9d-4f75-a1c1-794e7127531d response_code = 200 uri = /metrics/collect/47e44ab6-ee0b-4407-95b8-c3af199399c8
9142023-12-12T05:50:42.285ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 532 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:36542 req_id = 403815a4-1509-4729-b67e-da28ae32f112 response_code = 200 uri = /metrics/collect/47e44ab6-ee0b-4407-95b8-c3af199399c8