Max level shown:
12023-12-15T01:15:06.143ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 617 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:60064 req_id = 5b6ede51-1f7e-4708-8beb-e2ccdce866eb response_code = 200 uri = /metrics/collect/700e7b49-f840-40d5-92e8-83ab758ea8a7
22023-12-15T01:15:16.144ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 1466 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:60064 req_id = bcd993a2-528c-4ede-806a-6791b149d3f8 response_code = 200 uri = /metrics/collect/700e7b49-f840-40d5-92e8-83ab758ea8a7
32023-12-15T01:15:19.513ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:46101 req_id = d22e4674-a754-4420-a7d1-3578bb0b682b roles = RoleSet { roles: {} } uri = /nat/ipv4/changeset/0?limit=100
42023-12-15T01:15:19.513ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:46101 req_id = d22e4674-a754-4420-a7d1-3578bb0b682b resource = Database result = Ok(()) uri = /nat/ipv4/changeset/0?limit=100
52023-12-15T01:15:19.516ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 3442 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:46101 req_id = d22e4674-a754-4420-a7d1-3578bb0b682b response_code = 200 uri = /nat/ipv4/changeset/0?limit=100
62023-12-15T01:15:19.853ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 4 reason = Timeout
72023-12-15T01:15:19.854ZWARNnexus (BackgroundTasks): phantom disk task started background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:50
82023-12-15T01:15:19.854ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 4 reason = Timeout
92023-12-15T01:15:19.854ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
102023-12-15T01:15:19.854ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
112023-12-15T01:15:19.855ZDEBGnexus (DpdClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
122023-12-15T01:15:19.856ZDEBGnexus (DpdClient): client response result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "37e515fc-0381-47ab-94f4-2c6b205cc890", "content-length": "1", "date": "Fri, 15 Dec 2023 01:15:19 GMT"} })
132023-12-15T01:15:19.856ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
142023-12-15T01:15:19.856ZWARNnexus (BackgroundTasks): phantom disk task done background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:96
152023-12-15T01:15:19.856ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 2.900443ms iteration = 4
162023-12-15T01:15:19.856ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
172023-12-15T01:15:19.858ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 4.338855ms iteration = 4
182023-12-15T01:15:26.143ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 1248 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:60064 req_id = a05f6866-0dcd-4f77-bb69-fc82e1d88cee response_code = 200 uri = /metrics/collect/700e7b49-f840-40d5-92e8-83ab758ea8a7
192023-12-15T01:15:36.143ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 711 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:60064 req_id = 642997e4-36fa-483c-ad63-9d92bbea2142 response_code = 200 uri = /metrics/collect/700e7b49-f840-40d5-92e8-83ab758ea8a7
202023-12-15T01:15:46.143ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 688 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:60064 req_id = d7aabe96-e323-43d3-be87-888ce20a8094 response_code = 200 uri = /metrics/collect/700e7b49-f840-40d5-92e8-83ab758ea8a7
212023-12-15T01:15:49.547ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:46101 req_id = 14436ebf-a644-4d78-a94d-8cef3e18a51a roles = RoleSet { roles: {} } uri = /nat/ipv4/changeset/0?limit=100
222023-12-15T01:15:49.548ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:46101 req_id = 14436ebf-a644-4d78-a94d-8cef3e18a51a resource = Database result = Ok(()) uri = /nat/ipv4/changeset/0?limit=100
232023-12-15T01:15:49.550ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 2786 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:46101 req_id = 14436ebf-a644-4d78-a94d-8cef3e18a51a response_code = 200 uri = /nat/ipv4/changeset/0?limit=100
242023-12-15T01:15:49.849ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 4 reason = Timeout
252023-12-15T01:15:49.849ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
262023-12-15T01:15:49.849ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
272023-12-15T01:15:49.850ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
282023-12-15T01:15:49.850ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
292023-12-15T01:15:49.850ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
302023-12-15T01:15:49.851ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
312023-12-15T01:15:49.851ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 6 reason = Timeout
322023-12-15T01:15:49.851ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 4 reason = Timeout
332023-12-15T01:15:49.851ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
342023-12-15T01:15:49.851ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 4 reason = Timeout
352023-12-15T01:15:49.852ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_internal dns_group = internal roles = RoleSet { roles: {} }
362023-12-15T01:15:49.852ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 4 reason = Timeout
372023-12-15T01:15:49.852ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 4 reason = Timeout
382023-12-15T01:15:49.852ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
392023-12-15T01:15:49.852ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
402023-12-15T01:15:49.852ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 6 reason = Timeout
412023-12-15T01:15:49.852ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_external dns_group = external roles = RoleSet { roles: {} }
422023-12-15T01:15:49.852ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
432023-12-15T01:15:49.852ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_internal dns_group = internal resource = Database result = Ok(())
442023-12-15T01:15:49.852ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_external dns_group = external resource = Database result = Ok(())
452023-12-15T01:15:49.852ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
462023-12-15T01:15:49.853ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
472023-12-15T01:15:49.853ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
482023-12-15T01:15:49.853ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
492023-12-15T01:15:49.853ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
502023-12-15T01:15:49.854ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 5 reason = Timeout
512023-12-15T01:15:49.854ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
522023-12-15T01:15:49.854ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 5 reason = Timeout
532023-12-15T01:15:49.854ZWARNnexus (BackgroundTasks): phantom disk task started background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:50
542023-12-15T01:15:49.854ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
552023-12-15T01:15:49.854ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
562023-12-15T01:15:49.854ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
572023-12-15T01:15:49.855ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
582023-12-15T01:15:49.858ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_external dns_group = external resource = Fleet result = Ok(())
592023-12-15T01:15:49.858ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_external dns_group = external roles = RoleSet { roles: {} }
602023-12-15T01:15:49.858ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
612023-12-15T01:15:49.858ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
622023-12-15T01:15:49.858ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_external dns_group = external resource = Database result = Ok(())
632023-12-15T01:15:49.858ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
642023-12-15T01:15:49.859ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
652023-12-15T01:15:49.860ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_internal dns_group = internal resource = Fleet result = Ok(())
662023-12-15T01:15:49.860ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_internal dns_group = internal roles = RoleSet { roles: {} }
672023-12-15T01:15:49.860ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
682023-12-15T01:15:49.860ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
692023-12-15T01:15:49.860ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
702023-12-15T01:15:49.860ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
712023-12-15T01:15:49.860ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_internal dns_group = internal resource = Database result = Ok(())
722023-12-15T01:15:49.860ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
732023-12-15T01:15:49.861ZDEBGnexus (DpdClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
742023-12-15T01:15:49.861ZDEBGnexus (DpdClient): client response result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "25894892-aabc-4afc-acfb-120b6526dc37", "content-length": "1", "date": "Fri, 15 Dec 2023 01:15:49 GMT"} })
752023-12-15T01:15:49.861ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
762023-12-15T01:15:49.862ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
772023-12-15T01:15:49.865ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } dns_group = external
782023-12-15T01:15:49.865ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 13.122868ms iteration = 4
792023-12-15T01:15:49.865ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
802023-12-15T01:15:49.865ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
812023-12-15T01:15:49.865ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
822023-12-15T01:15:49.865ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
832023-12-15T01:15:49.865ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 11.668686ms iteration = 5
842023-12-15T01:15:49.865ZWARNnexus (BackgroundTasks): phantom disk task done background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:96
852023-12-15T01:15:49.865ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
862023-12-15T01:15:49.865ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 11.678056ms iteration = 5
872023-12-15T01:15:49.865ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
882023-12-15T01:15:49.866ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
892023-12-15T01:15:49.866ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
902023-12-15T01:15:49.866ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
912023-12-15T01:15:49.867ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } dns_group = internal
922023-12-15T01:15:49.867ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 15.600915ms iteration = 4
932023-12-15T01:15:49.867ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
942023-12-15T01:15:49.871ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
952023-12-15T01:15:49.871ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
962023-12-15T01:15:49.871ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
972023-12-15T01:15:49.871ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
982023-12-15T01:15:49.871ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
992023-12-15T01:15:49.871ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
1002023-12-15T01:15:49.872ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
1012023-12-15T01:15:49.872ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
1022023-12-15T01:15:49.872ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
1032023-12-15T01:15:49.872ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
1042023-12-15T01:15:49.872ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 1433420e-118d-4f4e-b816-04dd2f29c21e dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
1052023-12-15T01:15:49.872ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
1062023-12-15T01:15:49.873ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
1072023-12-15T01:15:49.873ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
1082023-12-15T01:15:49.873ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = b55d5eea-f93a-43bf-9796-c715ecea5842 dns_zone_name = oxide.test found_so_far = 0 version = 2
1092023-12-15T01:15:49.873ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
1102023-12-15T01:15:49.874ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
1112023-12-15T01:15:49.874ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1122023-12-15T01:15:49.874ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
1132023-12-15T01:15:49.875ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
1142023-12-15T01:15:49.875ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1152023-12-15T01:15:49.876ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1162023-12-15T01:15:49.878ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
1172023-12-15T01:15:49.878ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
1182023-12-15T01:15:49.879ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
1192023-12-15T01:15:49.879ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
1202023-12-15T01:15:49.879ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
1212023-12-15T01:15:49.879ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
1222023-12-15T01:15:49.879ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
1232023-12-15T01:15:49.879ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
1242023-12-15T01:15:49.880ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
1252023-12-15T01:15:49.881ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 1433420e-118d-4f4e-b816-04dd2f29c21e dns_zone_name = control-plane.oxide.internal found_so_far = 51 version = 1
1262023-12-15T01:15:49.881ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
1272023-12-15T01:15:49.881ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2023-12-15 01:14:03.961426 UTC dns_group = internal generation = 1
1282023-12-15T01:15:49.881ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 31.818049ms iteration = 4
1292023-12-15T01:15:49.881ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = b55d5eea-f93a-43bf-9796-c715ecea5842 dns_zone_name = oxide.test found_so_far = 1 version = 2
1302023-12-15T01:15:49.881ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
1312023-12-15T01:15:49.881ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2023-12-15 01:14:04.383795 UTC dns_group = external generation = 2
1322023-12-15T01:15:49.881ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 29.648747ms iteration = 4
1332023-12-15T01:15:49.892ZINFOnexus (BackgroundTasks): found Silo/DNS/TLS config (changed) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2023-12-15T01:13:49.851170Z, time_modified: 2023-12-15T01:13:49.851170Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: bcd56901-6236-45d5-99fd-58ae6b425f02, db_silo: Silo { identity: SiloIdentity { id: bcd56901-6236-45d5-99fd-58ae6b425f02, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2023-12-15T01:14:04.327942Z, time_modified: 2023-12-15T01:14:04.327942Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "6a0b318c733ecc8fe690fee9446b5a046a1e6779347663bbc3588532a4ce10cc" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: bcd56901-6236-45d5-99fd-58ae6b425f02, db_silo: Silo { identity: SiloIdentity { id: bcd56901-6236-45d5-99fd-58ae6b425f02, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2023-12-15T01:14:04.327942Z, time_modified: 2023-12-15T01:14:04.327942Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "6a0b318c733ecc8fe690fee9446b5a046a1e6779347663bbc3588532a4ce10cc" }] }) } file = nexus/src/app/background/external_endpoints.rs:105
1342023-12-15T01:15:49.892ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 40.208406ms iteration = 4
1352023-12-15T01:15:49.921ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
1362023-12-15T01:15:49.928ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
1372023-12-15T01:15:49.987ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
1382023-12-15T01:15:49.987ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::9)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "633ddb52-54f6-4e6e-a23b-f4bae61e5a44", "date": "Fri, 15 Dec 2023 01:15:49 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
1392023-12-15T01:15:49.987ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/dns_propagation.rs:173 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
1402023-12-15T01:15:49.988ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::8)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "3faa2bb8-25f7-45b3-b34f-c5a9a5db9ebc", "date": "Fri, 15 Dec 2023 01:15:49 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
1412023-12-15T01:15:49.988ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/dns_propagation.rs:173 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
1422023-12-15T01:15:49.990ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/dns_propagation.rs:110 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
1432023-12-15T01:15:49.990ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 138.852611ms iteration = 6
1442023-12-15T01:15:49.993ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
1452023-12-15T01:15:50.024ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
1462023-12-15T01:15:50.025ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:3::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "081e7648-5caa-4da2-8d93-6cad2ac43b95", "date": "Fri, 15 Dec 2023 01:15:50 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
1472023-12-15T01:15:50.025ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/dns_propagation.rs:173 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
1482023-12-15T01:15:50.026ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:1::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "bb1cb106-cba1-4601-83a1-bad5c3bb6795", "date": "Fri, 15 Dec 2023 01:15:50 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
1492023-12-15T01:15:50.026ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/dns_propagation.rs:173 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
1502023-12-15T01:15:50.027ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:2::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "6b7c3f2e-8beb-471a-9ce5-722e5b8b0eb1", "date": "Fri, 15 Dec 2023 01:15:50 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
1512023-12-15T01:15:50.027ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/dns_propagation.rs:173 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
1522023-12-15T01:15:50.028ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/dns_propagation.rs:110 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
1532023-12-15T01:15:50.028ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 176.907574ms iteration = 6
1542023-12-15T01:15:56.142ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 623 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:60064 req_id = bc47c2e7-5888-46c1-ad30-01448704069e response_code = 200 uri = /metrics/collect/700e7b49-f840-40d5-92e8-83ab758ea8a7
1552023-12-15T01:16:06.143ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 718 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:60064 req_id = 10a5f9a0-27bb-4943-a70b-1d3c854e4407 response_code = 200 uri = /metrics/collect/700e7b49-f840-40d5-92e8-83ab758ea8a7
1562023-12-15T01:16:16.143ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 718 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:60064 req_id = f1a2dec2-77fc-4b0d-ad4f-ddd01e5420a3 response_code = 200 uri = /metrics/collect/700e7b49-f840-40d5-92e8-83ab758ea8a7
1572023-12-15T01:16:19.580ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:46101 req_id = 57a704b1-1f62-4b1e-b58b-c522b428aaba roles = RoleSet { roles: {} } uri = /nat/ipv4/changeset/0?limit=100
1582023-12-15T01:16:19.580ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:46101 req_id = 57a704b1-1f62-4b1e-b58b-c522b428aaba resource = Database result = Ok(()) uri = /nat/ipv4/changeset/0?limit=100
1592023-12-15T01:16:19.583ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 3332 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:46101 req_id = 57a704b1-1f62-4b1e-b58b-c522b428aaba response_code = 200 uri = /nat/ipv4/changeset/0?limit=100
1602023-12-15T01:16:19.854ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 6 reason = Timeout
1612023-12-15T01:16:19.854ZWARNnexus (BackgroundTasks): phantom disk task started background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:50
1622023-12-15T01:16:19.854ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 6 reason = Timeout
1632023-12-15T01:16:19.854ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
1642023-12-15T01:16:19.854ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
1652023-12-15T01:16:19.856ZDEBGnexus (DpdClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
1662023-12-15T01:16:19.856ZDEBGnexus (DpdClient): client response result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "5c6b8ff4-fae8-44cc-93c6-605cd5e5ba1c", "content-length": "1", "date": "Fri, 15 Dec 2023 01:16:19 GMT"} })
1672023-12-15T01:16:19.856ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
1682023-12-15T01:16:19.856ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
1692023-12-15T01:16:19.857ZWARNnexus (BackgroundTasks): phantom disk task done background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:96
1702023-12-15T01:16:19.857ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.468132ms iteration = 6
1712023-12-15T01:16:19.858ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 4.02183ms iteration = 6
1722023-12-15T01:16:26.143ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 869 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:60064 req_id = 6aeb9ad7-ec58-4ff4-b6c4-840528828307 response_code = 200 uri = /metrics/collect/700e7b49-f840-40d5-92e8-83ab758ea8a7
1732023-12-15T01:16:36.143ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 600 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:60064 req_id = 81f12d2a-c0ff-4026-98fe-c1911c79ead8 response_code = 200 uri = /metrics/collect/700e7b49-f840-40d5-92e8-83ab758ea8a7
1742023-12-15T01:16:46.143ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 1102 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:60064 req_id = 0a4dd946-cc1e-4049-b3c0-097697effab3 response_code = 200 uri = /metrics/collect/700e7b49-f840-40d5-92e8-83ab758ea8a7
1752023-12-15T01:16:49.614ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:46101 req_id = 3f141bc3-3621-4e50-b9d4-df1ba0f475a2 roles = RoleSet { roles: {} } uri = /nat/ipv4/changeset/0?limit=100
1762023-12-15T01:16:49.614ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:46101 req_id = 3f141bc3-3621-4e50-b9d4-df1ba0f475a2 resource = Database result = Ok(()) uri = /nat/ipv4/changeset/0?limit=100
1772023-12-15T01:16:49.617ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 2878 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:46101 req_id = 3f141bc3-3621-4e50-b9d4-df1ba0f475a2 response_code = 200 uri = /nat/ipv4/changeset/0?limit=100
1782023-12-15T01:16:49.850ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 5 reason = Timeout
1792023-12-15T01:16:49.850ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
1802023-12-15T01:16:49.850ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
1812023-12-15T01:16:49.850ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
1822023-12-15T01:16:49.851ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
1832023-12-15T01:16:49.851ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
1842023-12-15T01:16:49.851ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 5 reason = Timeout
1852023-12-15T01:16:49.851ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_internal dns_group = internal roles = RoleSet { roles: {} }
1862023-12-15T01:16:49.851ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 7 reason = Timeout
1872023-12-15T01:16:49.851ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 5 reason = Timeout
1882023-12-15T01:16:49.851ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 5 reason = Timeout
1892023-12-15T01:16:49.851ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_external dns_group = external roles = RoleSet { roles: {} }
1902023-12-15T01:16:49.851ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 5 reason = Timeout
1912023-12-15T01:16:49.851ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
1922023-12-15T01:16:49.851ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
1932023-12-15T01:16:49.851ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 7 reason = Timeout
1942023-12-15T01:16:49.851ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
1952023-12-15T01:16:49.852ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_internal dns_group = internal resource = Database result = Ok(())
1962023-12-15T01:16:49.852ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
1972023-12-15T01:16:49.852ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_external dns_group = external resource = Database result = Ok(())
1982023-12-15T01:16:49.852ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
1992023-12-15T01:16:49.852ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2002023-12-15T01:16:49.853ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
2012023-12-15T01:16:49.853ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
2022023-12-15T01:16:49.853ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2032023-12-15T01:16:49.853ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2042023-12-15T01:16:49.853ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 7 reason = Timeout
2052023-12-15T01:16:49.853ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 7 reason = Timeout
2062023-12-15T01:16:49.853ZWARNnexus (BackgroundTasks): phantom disk task started background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:50
2072023-12-15T01:16:49.853ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
2082023-12-15T01:16:49.854ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
2092023-12-15T01:16:49.854ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2102023-12-15T01:16:49.855ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2112023-12-15T01:16:49.855ZDEBGnexus (DpdClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
2122023-12-15T01:16:49.855ZDEBGnexus (DpdClient): client response result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "c16ba837-b545-4fb1-9498-ce69e2c2d8b1", "content-length": "1", "date": "Fri, 15 Dec 2023 01:16:49 GMT"} })
2132023-12-15T01:16:49.855ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
2142023-12-15T01:16:49.855ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
2152023-12-15T01:16:49.856ZWARNnexus (BackgroundTasks): phantom disk task done background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:96
2162023-12-15T01:16:49.856ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 2.280044ms iteration = 7
2172023-12-15T01:16:49.857ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 3.496133ms iteration = 7
2182023-12-15T01:16:49.858ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_external dns_group = external resource = Fleet result = Ok(())
2192023-12-15T01:16:49.858ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_external dns_group = external roles = RoleSet { roles: {} }
2202023-12-15T01:16:49.858ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_external dns_group = external resource = Database result = Ok(())
2212023-12-15T01:16:49.858ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_internal dns_group = internal resource = Fleet result = Ok(())
2222023-12-15T01:16:49.858ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_internal dns_group = internal roles = RoleSet { roles: {} }
2232023-12-15T01:16:49.858ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_internal dns_group = internal resource = Database result = Ok(())
2242023-12-15T01:16:49.859ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
2252023-12-15T01:16:49.859ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
2262023-12-15T01:16:49.859ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
2272023-12-15T01:16:49.859ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
2282023-12-15T01:16:49.860ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
2292023-12-15T01:16:49.860ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } dns_group = external
2302023-12-15T01:16:49.860ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 8.654651ms iteration = 5
2312023-12-15T01:16:49.860ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
2322023-12-15T01:16:49.860ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
2332023-12-15T01:16:49.860ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
2342023-12-15T01:16:49.860ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
2352023-12-15T01:16:49.860ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
2362023-12-15T01:16:49.860ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
2372023-12-15T01:16:49.860ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
2382023-12-15T01:16:49.860ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
2392023-12-15T01:16:49.860ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } dns_group = internal
2402023-12-15T01:16:49.860ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 9.157678ms iteration = 5
2412023-12-15T01:16:49.860ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
2422023-12-15T01:16:49.861ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
2432023-12-15T01:16:49.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
2442023-12-15T01:16:49.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
2452023-12-15T01:16:49.862ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2462023-12-15T01:16:49.862ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2472023-12-15T01:16:49.862ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2482023-12-15T01:16:49.866ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
2492023-12-15T01:16:49.866ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
2502023-12-15T01:16:49.866ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
2512023-12-15T01:16:49.866ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
2522023-12-15T01:16:49.866ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
2532023-12-15T01:16:49.866ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
2542023-12-15T01:16:49.867ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
2552023-12-15T01:16:49.867ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
2562023-12-15T01:16:49.867ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
2572023-12-15T01:16:49.867ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
2582023-12-15T01:16:49.867ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 1433420e-118d-4f4e-b816-04dd2f29c21e dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
2592023-12-15T01:16:49.867ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
2602023-12-15T01:16:49.867ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
2612023-12-15T01:16:49.867ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = b55d5eea-f93a-43bf-9796-c715ecea5842 dns_zone_name = oxide.test found_so_far = 0 version = 2
2622023-12-15T01:16:49.867ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
2632023-12-15T01:16:49.867ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
2642023-12-15T01:16:49.868ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
2652023-12-15T01:16:49.868ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2662023-12-15T01:16:49.868ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
2672023-12-15T01:16:49.869ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2682023-12-15T01:16:49.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
2692023-12-15T01:16:49.870ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2702023-12-15T01:16:49.873ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
2712023-12-15T01:16:49.873ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
2722023-12-15T01:16:49.873ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
2732023-12-15T01:16:49.873ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
2742023-12-15T01:16:49.873ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
2752023-12-15T01:16:49.873ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
2762023-12-15T01:16:49.873ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
2772023-12-15T01:16:49.873ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
2782023-12-15T01:16:49.874ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
2792023-12-15T01:16:49.874ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = b55d5eea-f93a-43bf-9796-c715ecea5842 dns_zone_name = oxide.test found_so_far = 1 version = 2
2802023-12-15T01:16:49.874ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
2812023-12-15T01:16:49.874ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2023-12-15 01:14:04.383795 UTC dns_group = external generation = 2
2822023-12-15T01:16:49.874ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 22.55523ms iteration = 5
2832023-12-15T01:16:49.875ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 1433420e-118d-4f4e-b816-04dd2f29c21e dns_zone_name = control-plane.oxide.internal found_so_far = 51 version = 1
2842023-12-15T01:16:49.875ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
2852023-12-15T01:16:49.875ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2023-12-15 01:14:03.961426 UTC dns_group = internal generation = 1
2862023-12-15T01:16:49.875ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 25.273751ms iteration = 5
2872023-12-15T01:16:49.888ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2023-12-15T01:13:49.851170Z, time_modified: 2023-12-15T01:13:49.851170Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: bcd56901-6236-45d5-99fd-58ae6b425f02, db_silo: Silo { identity: SiloIdentity { id: bcd56901-6236-45d5-99fd-58ae6b425f02, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2023-12-15T01:14:04.327942Z, time_modified: 2023-12-15T01:14:04.327942Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "6a0b318c733ecc8fe690fee9446b5a046a1e6779347663bbc3588532a4ce10cc" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: bcd56901-6236-45d5-99fd-58ae6b425f02, db_silo: Silo { identity: SiloIdentity { id: bcd56901-6236-45d5-99fd-58ae6b425f02, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2023-12-15T01:14:04.327942Z, time_modified: 2023-12-15T01:14:04.327942Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "6a0b318c733ecc8fe690fee9446b5a046a1e6779347663bbc3588532a4ce10cc" }] }) }
2882023-12-15T01:16:49.888ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 37.089428ms iteration = 5
2892023-12-15T01:16:49.991ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
2902023-12-15T01:16:49.993ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
2912023-12-15T01:16:50.111ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
2922023-12-15T01:16:50.124ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
2932023-12-15T01:16:50.125ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::8)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "ba410009-89f5-49ba-a0f1-f0d7809bc373", "date": "Fri, 15 Dec 2023 01:16:49 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
2942023-12-15T01:16:50.125ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/dns_propagation.rs:173 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
2952023-12-15T01:16:50.125ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::9)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "f9365856-9d48-48c9-adfb-d0f28cc8dd5f", "date": "Fri, 15 Dec 2023 01:16:49 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
2962023-12-15T01:16:50.125ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/dns_propagation.rs:173 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
2972023-12-15T01:16:50.127ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/dns_propagation.rs:110 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
2982023-12-15T01:16:50.127ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 276.002317ms iteration = 7
2992023-12-15T01:16:50.151ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
3002023-12-15T01:16:50.151ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:3::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "185801d6-c7fd-42cf-92ad-7ebde56acefc", "date": "Fri, 15 Dec 2023 01:16:49 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
3012023-12-15T01:16:50.151ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/dns_propagation.rs:173 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
3022023-12-15T01:16:50.153ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:1::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "2fe535a7-de08-4372-9bb8-a144c695980a", "date": "Fri, 15 Dec 2023 01:16:49 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
3032023-12-15T01:16:50.153ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/dns_propagation.rs:173 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
3042023-12-15T01:16:50.154ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:2::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "8bf5168b-7db9-423e-a870-c5228ae9e5a4", "date": "Fri, 15 Dec 2023 01:16:49 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
3052023-12-15T01:16:50.154ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/dns_propagation.rs:173 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
3062023-12-15T01:16:50.155ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/dns_propagation.rs:110 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
3072023-12-15T01:16:50.155ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 303.819045ms iteration = 7
3082023-12-15T01:16:56.143ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 1020 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:60064 req_id = fd4ce1b0-30f1-46e2-85a3-0c12c91b46f6 response_code = 200 uri = /metrics/collect/700e7b49-f840-40d5-92e8-83ab758ea8a7
3092023-12-15T01:17:06.143ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 750 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:60064 req_id = 846c5ff0-4ea9-4521-9218-56a0299f7b90 response_code = 200 uri = /metrics/collect/700e7b49-f840-40d5-92e8-83ab758ea8a7
3102023-12-15T01:17:16.144ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 1433 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:60064 req_id = 2efb277f-7055-44a7-9f9d-134a28471dfd response_code = 200 uri = /metrics/collect/700e7b49-f840-40d5-92e8-83ab758ea8a7
3112023-12-15T01:17:19.648ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:46101 req_id = d7b242f0-2449-4715-b131-7a248559b7a7 roles = RoleSet { roles: {} } uri = /nat/ipv4/changeset/0?limit=100
3122023-12-15T01:17:19.648ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:46101 req_id = d7b242f0-2449-4715-b131-7a248559b7a7 resource = Database result = Ok(()) uri = /nat/ipv4/changeset/0?limit=100
3132023-12-15T01:17:19.651ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 3059 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:46101 req_id = d7b242f0-2449-4715-b131-7a248559b7a7 response_code = 200 uri = /nat/ipv4/changeset/0?limit=100
3142023-12-15T01:17:19.853ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 8 reason = Timeout
3152023-12-15T01:17:19.853ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
3162023-12-15T01:17:19.853ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 8 reason = Timeout
3172023-12-15T01:17:19.853ZWARNnexus (BackgroundTasks): phantom disk task started background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:50
3182023-12-15T01:17:19.853ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
3192023-12-15T01:17:19.855ZDEBGnexus (DpdClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
3202023-12-15T01:17:19.855ZDEBGnexus (DpdClient): client response result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "150e6256-ac71-4392-9ddd-e12ca59693d1", "content-length": "1", "date": "Fri, 15 Dec 2023 01:17:19 GMT"} })
3212023-12-15T01:17:19.855ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
3222023-12-15T01:17:19.855ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
3232023-12-15T01:17:19.855ZWARNnexus (BackgroundTasks): phantom disk task done background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:96
3242023-12-15T01:17:19.855ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 2.347915ms iteration = 8
3252023-12-15T01:17:19.856ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 3.33258ms iteration = 8
3262023-12-15T01:17:26.144ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 1846 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:60064 req_id = a80e3b85-cfe3-4f17-9479-921b548be56f response_code = 200 uri = /metrics/collect/700e7b49-f840-40d5-92e8-83ab758ea8a7
3272023-12-15T01:17:36.143ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 1270 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:60064 req_id = 206b8e14-b9b0-4ac0-a740-6567d40c7429 response_code = 200 uri = /metrics/collect/700e7b49-f840-40d5-92e8-83ab758ea8a7
3282023-12-15T01:17:46.143ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 862 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:60064 req_id = 97e9be15-2c00-4c6d-be60-95cabd6c138a response_code = 200 uri = /metrics/collect/700e7b49-f840-40d5-92e8-83ab758ea8a7
3292023-12-15T01:17:49.683ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:46101 req_id = 1b34470f-469a-4e42-96b8-07830fb5f6ce roles = RoleSet { roles: {} } uri = /nat/ipv4/changeset/0?limit=100
3302023-12-15T01:17:49.683ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:46101 req_id = 1b34470f-469a-4e42-96b8-07830fb5f6ce resource = Database result = Ok(()) uri = /nat/ipv4/changeset/0?limit=100
3312023-12-15T01:17:49.685ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 3457 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:46101 req_id = 1b34470f-469a-4e42-96b8-07830fb5f6ce response_code = 200 uri = /nat/ipv4/changeset/0?limit=100
3322023-12-15T01:17:49.849ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 6 reason = Timeout
3332023-12-15T01:17:49.849ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
3342023-12-15T01:17:49.850ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
3352023-12-15T01:17:49.850ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
3362023-12-15T01:17:49.850ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
3372023-12-15T01:17:49.850ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
3382023-12-15T01:17:49.851ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3392023-12-15T01:17:49.851ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 8 reason = Timeout
3402023-12-15T01:17:49.851ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 6 reason = Timeout
3412023-12-15T01:17:49.851ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 6 reason = Timeout
3422023-12-15T01:17:49.851ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
3432023-12-15T01:17:49.851ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
3442023-12-15T01:17:49.851ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 6 reason = Timeout
3452023-12-15T01:17:49.851ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_external dns_group = external roles = RoleSet { roles: {} }
3462023-12-15T01:17:49.852ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 6 reason = Timeout
3472023-12-15T01:17:49.852ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_internal dns_group = internal roles = RoleSet { roles: {} }
3482023-12-15T01:17:49.852ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
3492023-12-15T01:17:49.852ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 8 reason = Timeout
3502023-12-15T01:17:49.852ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_internal dns_group = internal resource = Database result = Ok(())
3512023-12-15T01:17:49.852ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
3522023-12-15T01:17:49.852ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_external dns_group = external resource = Database result = Ok(())
3532023-12-15T01:17:49.852ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
3542023-12-15T01:17:49.852ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
3552023-12-15T01:17:49.853ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
3562023-12-15T01:17:49.853ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3572023-12-15T01:17:49.853ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3582023-12-15T01:17:49.854ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 9 reason = Timeout
3592023-12-15T01:17:49.854ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 9 reason = Timeout
3602023-12-15T01:17:49.854ZWARNnexus (BackgroundTasks): phantom disk task started background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:50
3612023-12-15T01:17:49.854ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
3622023-12-15T01:17:49.854ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3632023-12-15T01:17:49.854ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
3642023-12-15T01:17:49.854ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3652023-12-15T01:17:49.855ZDEBGnexus (DpdClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
3662023-12-15T01:17:49.855ZDEBGnexus (DpdClient): client response result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "e90b4a23-d83b-4918-90ef-07fadaa434ac", "content-length": "1", "date": "Fri, 15 Dec 2023 01:17:49 GMT"} })
3672023-12-15T01:17:49.855ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
3682023-12-15T01:17:49.856ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
3692023-12-15T01:17:49.856ZWARNnexus (BackgroundTasks): phantom disk task done background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:96
3702023-12-15T01:17:49.856ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 2.339715ms iteration = 9
3712023-12-15T01:17:49.856ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
3722023-12-15T01:17:49.857ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
3732023-12-15T01:17:49.857ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
3742023-12-15T01:17:49.857ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
3752023-12-15T01:17:49.857ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
3762023-12-15T01:17:49.857ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 3.657635ms iteration = 9
3772023-12-15T01:17:49.857ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
3782023-12-15T01:17:49.858ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_external dns_group = external resource = Fleet result = Ok(())
3792023-12-15T01:17:49.858ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_external dns_group = external roles = RoleSet { roles: {} }
3802023-12-15T01:17:49.858ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
3812023-12-15T01:17:49.858ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
3822023-12-15T01:17:49.858ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_internal dns_group = internal resource = Fleet result = Ok(())
3832023-12-15T01:17:49.858ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_internal dns_group = internal roles = RoleSet { roles: {} }
3842023-12-15T01:17:49.858ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_external dns_group = external resource = Database result = Ok(())
3852023-12-15T01:17:49.859ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3862023-12-15T01:17:49.859ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
3872023-12-15T01:17:49.859ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_internal dns_group = internal resource = Database result = Ok(())
3882023-12-15T01:17:49.859ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
3892023-12-15T01:17:49.860ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
3902023-12-15T01:17:49.860ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } dns_group = external
3912023-12-15T01:17:49.860ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 8.567749ms iteration = 6
3922023-12-15T01:17:49.860ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
3932023-12-15T01:17:49.860ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } dns_group = internal
3942023-12-15T01:17:49.860ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 8.745721ms iteration = 6
3952023-12-15T01:17:49.861ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3962023-12-15T01:17:49.861ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
3972023-12-15T01:17:49.861ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
3982023-12-15T01:17:49.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
3992023-12-15T01:17:49.863ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
4002023-12-15T01:17:49.863ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 1433420e-118d-4f4e-b816-04dd2f29c21e dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
4012023-12-15T01:17:49.863ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
4022023-12-15T01:17:49.863ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
4032023-12-15T01:17:49.864ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
4042023-12-15T01:17:49.864ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
4052023-12-15T01:17:49.864ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4062023-12-15T01:17:49.864ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
4072023-12-15T01:17:49.866ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
4082023-12-15T01:17:49.866ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
4092023-12-15T01:17:49.867ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
4102023-12-15T01:17:49.867ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
4112023-12-15T01:17:49.867ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4122023-12-15T01:17:49.867ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
4132023-12-15T01:17:49.869ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
4142023-12-15T01:17:49.869ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
4152023-12-15T01:17:49.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
4162023-12-15T01:17:49.869ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 1433420e-118d-4f4e-b816-04dd2f29c21e dns_zone_name = control-plane.oxide.internal found_so_far = 51 version = 1
4172023-12-15T01:17:49.869ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
4182023-12-15T01:17:49.869ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2023-12-15 01:14:03.961426 UTC dns_group = internal generation = 1
4192023-12-15T01:17:49.869ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 19.668167ms iteration = 6
4202023-12-15T01:17:49.875ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2023-12-15T01:13:49.851170Z, time_modified: 2023-12-15T01:13:49.851170Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: bcd56901-6236-45d5-99fd-58ae6b425f02, db_silo: Silo { identity: SiloIdentity { id: bcd56901-6236-45d5-99fd-58ae6b425f02, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2023-12-15T01:14:04.327942Z, time_modified: 2023-12-15T01:14:04.327942Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "6a0b318c733ecc8fe690fee9446b5a046a1e6779347663bbc3588532a4ce10cc" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: bcd56901-6236-45d5-99fd-58ae6b425f02, db_silo: Silo { identity: SiloIdentity { id: bcd56901-6236-45d5-99fd-58ae6b425f02, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2023-12-15T01:14:04.327942Z, time_modified: 2023-12-15T01:14:04.327942Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "6a0b318c733ecc8fe690fee9446b5a046a1e6779347663bbc3588532a4ce10cc" }] }) }
4212023-12-15T01:17:49.875ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 23.546444ms iteration = 6
4222023-12-15T01:17:49.898ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
4232023-12-15T01:17:49.898ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
4242023-12-15T01:17:49.898ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
4252023-12-15T01:17:49.898ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
4262023-12-15T01:17:49.898ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
4272023-12-15T01:17:49.899ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4282023-12-15T01:17:49.902ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
4292023-12-15T01:17:49.902ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
4302023-12-15T01:17:49.902ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
4312023-12-15T01:17:49.904ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
4322023-12-15T01:17:49.904ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = b55d5eea-f93a-43bf-9796-c715ecea5842 dns_zone_name = oxide.test found_so_far = 0 version = 2
4332023-12-15T01:17:49.904ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
4342023-12-15T01:17:49.904ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
4352023-12-15T01:17:49.905ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4362023-12-15T01:17:49.908ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
4372023-12-15T01:17:49.908ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
4382023-12-15T01:17:49.908ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
4392023-12-15T01:17:49.909ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = b55d5eea-f93a-43bf-9796-c715ecea5842 dns_zone_name = oxide.test found_so_far = 1 version = 2
4402023-12-15T01:17:49.909ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
4412023-12-15T01:17:49.909ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2023-12-15 01:14:04.383795 UTC dns_group = external generation = 2
4422023-12-15T01:17:49.909ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 57.982573ms iteration = 6
4432023-12-15T01:17:49.916ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
4442023-12-15T01:17:49.917ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
4452023-12-15T01:17:49.977ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
4462023-12-15T01:17:49.979ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
4472023-12-15T01:17:49.980ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::8)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "828ec178-b31b-49bf-8d40-8a57839b5de1", "date": "Fri, 15 Dec 2023 01:17:49 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
4482023-12-15T01:17:49.980ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/dns_propagation.rs:173 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
4492023-12-15T01:17:49.981ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::9)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "c744fb1e-fccf-47f4-8bf1-bda9b8e598f2", "date": "Fri, 15 Dec 2023 01:17:49 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
4502023-12-15T01:17:49.981ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/dns_propagation.rs:173 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
4512023-12-15T01:17:49.982ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/dns_propagation.rs:110 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
4522023-12-15T01:17:49.982ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 130.348753ms iteration = 8
4532023-12-15T01:17:50.010ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
4542023-12-15T01:17:50.010ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:2::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "2aa93a27-3d1d-46f1-bd56-7d51aea759c4", "date": "Fri, 15 Dec 2023 01:17:49 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
4552023-12-15T01:17:50.010ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/dns_propagation.rs:173 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
4562023-12-15T01:17:50.012ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:3::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "7d7dc669-f23f-4647-8bf3-268a837a1bdf", "date": "Fri, 15 Dec 2023 01:17:49 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
4572023-12-15T01:17:50.012ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/dns_propagation.rs:173 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
4582023-12-15T01:17:50.013ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:1::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "b72a38c2-1733-478e-b1e5-b1bd82de54b0", "date": "Fri, 15 Dec 2023 01:17:49 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
4592023-12-15T01:17:50.013ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/dns_propagation.rs:173 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
4602023-12-15T01:17:50.015ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/dns_propagation.rs:110 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
4612023-12-15T01:17:50.015ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 163.649544ms iteration = 8
4622023-12-15T01:17:56.143ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 616 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:60064 req_id = 98433545-eb7d-459b-8a4d-b6510504a43f response_code = 200 uri = /metrics/collect/700e7b49-f840-40d5-92e8-83ab758ea8a7
4632023-12-15T01:18:06.143ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 692 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:60064 req_id = 7a227837-59dc-4f85-8174-419f467f41fb response_code = 200 uri = /metrics/collect/700e7b49-f840-40d5-92e8-83ab758ea8a7
4642023-12-15T01:18:16.143ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 615 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:60064 req_id = 988f7609-b360-4254-a206-2865fd917f22 response_code = 200 uri = /metrics/collect/700e7b49-f840-40d5-92e8-83ab758ea8a7
4652023-12-15T01:18:19.717ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:46101 req_id = c990a43f-67f8-41e7-a4db-b6c647adbc35 roles = RoleSet { roles: {} } uri = /nat/ipv4/changeset/0?limit=100
4662023-12-15T01:18:19.717ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:46101 req_id = c990a43f-67f8-41e7-a4db-b6c647adbc35 resource = Database result = Ok(()) uri = /nat/ipv4/changeset/0?limit=100
4672023-12-15T01:18:19.720ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 2838 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:46101 req_id = c990a43f-67f8-41e7-a4db-b6c647adbc35 response_code = 200 uri = /nat/ipv4/changeset/0?limit=100
4682023-12-15T01:18:19.854ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 10 reason = Timeout
4692023-12-15T01:18:19.854ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 10 reason = Timeout
4702023-12-15T01:18:19.854ZWARNnexus (BackgroundTasks): phantom disk task started background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:50
4712023-12-15T01:18:19.854ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
4722023-12-15T01:18:19.854ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
4732023-12-15T01:18:19.855ZDEBGnexus (DpdClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
4742023-12-15T01:18:19.856ZDEBGnexus (DpdClient): client response result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "03dec717-d317-43a2-9d66-a9ef842b9e37", "content-length": "1", "date": "Fri, 15 Dec 2023 01:18:19 GMT"} })
4752023-12-15T01:18:19.856ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
4762023-12-15T01:18:19.856ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
4772023-12-15T01:18:19.856ZWARNnexus (BackgroundTasks): phantom disk task done background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:96
4782023-12-15T01:18:19.857ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 2.788022ms iteration = 10
4792023-12-15T01:18:19.857ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 3.457972ms iteration = 10
4802023-12-15T01:18:26.144ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 977 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:60064 req_id = 9eb98d1b-0276-4ae5-9c77-f62207579304 response_code = 200 uri = /metrics/collect/700e7b49-f840-40d5-92e8-83ab758ea8a7
4812023-12-15T01:18:36.144ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 1036 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:60064 req_id = 20454f82-7e07-42c5-a13d-05ba697980b6 response_code = 200 uri = /metrics/collect/700e7b49-f840-40d5-92e8-83ab758ea8a7
4822023-12-15T01:18:46.144ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 723 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:60064 req_id = b4311273-39b6-4fff-9216-e992aed392c3 response_code = 200 uri = /metrics/collect/700e7b49-f840-40d5-92e8-83ab758ea8a7
4832023-12-15T01:18:49.752ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:46101 req_id = e8fb3914-d930-4fa8-badb-580448c799a7 roles = RoleSet { roles: {} } uri = /nat/ipv4/changeset/0?limit=100
4842023-12-15T01:18:49.753ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:46101 req_id = e8fb3914-d930-4fa8-badb-580448c799a7 resource = Database result = Ok(()) uri = /nat/ipv4/changeset/0?limit=100
4852023-12-15T01:18:49.755ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 4069 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:46101 req_id = e8fb3914-d930-4fa8-badb-580448c799a7 response_code = 200 uri = /nat/ipv4/changeset/0?limit=100
4862023-12-15T01:18:49.849ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 7 reason = Timeout
4872023-12-15T01:18:49.849ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
4882023-12-15T01:18:49.849ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
4892023-12-15T01:18:49.850ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
4902023-12-15T01:18:49.850ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
4912023-12-15T01:18:49.850ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
4922023-12-15T01:18:49.851ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 9 reason = Timeout
4932023-12-15T01:18:49.851ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 7 reason = Timeout
4942023-12-15T01:18:49.851ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
4952023-12-15T01:18:49.851ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4962023-12-15T01:18:49.851ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 7 reason = Timeout
4972023-12-15T01:18:49.851ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 7 reason = Timeout
4982023-12-15T01:18:49.851ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_internal dns_group = internal roles = RoleSet { roles: {} }
4992023-12-15T01:18:49.851ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 7 reason = Timeout
5002023-12-15T01:18:49.851ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_external dns_group = external roles = RoleSet { roles: {} }
5012023-12-15T01:18:49.851ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
5022023-12-15T01:18:49.852ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 9 reason = Timeout
5032023-12-15T01:18:49.852ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
5042023-12-15T01:18:49.852ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
5052023-12-15T01:18:49.852ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_internal dns_group = internal resource = Database result = Ok(())
5062023-12-15T01:18:49.852ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_external dns_group = external resource = Database result = Ok(())
5072023-12-15T01:18:49.852ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
5082023-12-15T01:18:49.853ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
5092023-12-15T01:18:49.853ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5102023-12-15T01:18:49.853ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
5112023-12-15T01:18:49.853ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5122023-12-15T01:18:49.853ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5132023-12-15T01:18:49.854ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 11 reason = Timeout
5142023-12-15T01:18:49.854ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
5152023-12-15T01:18:49.854ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 11 reason = Timeout
5162023-12-15T01:18:49.854ZWARNnexus (BackgroundTasks): phantom disk task started background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:50
5172023-12-15T01:18:49.854ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
5182023-12-15T01:18:49.854ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5192023-12-15T01:18:49.855ZDEBGnexus (DpdClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
5202023-12-15T01:18:49.855ZDEBGnexus (DpdClient): client response result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "650238c8-23b8-4204-8a0e-419e05d1a282", "content-length": "1", "date": "Fri, 15 Dec 2023 01:18:49 GMT"} })
5212023-12-15T01:18:49.855ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
5222023-12-15T01:18:49.856ZWARNnexus (BackgroundTasks): phantom disk task done background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:96
5232023-12-15T01:18:49.856ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 2.281505ms iteration = 11
5242023-12-15T01:18:49.856ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
5252023-12-15T01:18:49.857ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
5262023-12-15T01:18:49.857ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 3.524913ms iteration = 11
5272023-12-15T01:18:49.857ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
5282023-12-15T01:18:49.857ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
5292023-12-15T01:18:49.858ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
5302023-12-15T01:18:49.858ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_external dns_group = external resource = Fleet result = Ok(())
5312023-12-15T01:18:49.858ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_external dns_group = external roles = RoleSet { roles: {} }
5322023-12-15T01:18:49.858ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
5332023-12-15T01:18:49.858ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
5342023-12-15T01:18:49.858ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
5352023-12-15T01:18:49.858ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
5362023-12-15T01:18:49.858ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_external dns_group = external resource = Database result = Ok(())
5372023-12-15T01:18:49.858ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_internal dns_group = internal resource = Fleet result = Ok(())
5382023-12-15T01:18:49.858ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_internal dns_group = internal roles = RoleSet { roles: {} }
5392023-12-15T01:18:49.858ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
5402023-12-15T01:18:49.858ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_internal dns_group = internal resource = Database result = Ok(())
5412023-12-15T01:18:49.859ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
5422023-12-15T01:18:49.859ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5432023-12-15T01:18:49.859ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
5442023-12-15T01:18:49.859ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } dns_group = external
5452023-12-15T01:18:49.859ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 7.887119ms iteration = 7
5462023-12-15T01:18:49.859ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
5472023-12-15T01:18:49.860ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } dns_group = internal
5482023-12-15T01:18:49.860ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 8.545739ms iteration = 7
5492023-12-15T01:18:49.860ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5502023-12-15T01:18:49.860ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
5512023-12-15T01:18:49.860ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
5522023-12-15T01:18:49.860ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
5532023-12-15T01:18:49.860ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
5542023-12-15T01:18:49.860ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
5552023-12-15T01:18:49.862ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5562023-12-15T01:18:49.863ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
5572023-12-15T01:18:49.863ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
5582023-12-15T01:18:49.863ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
5592023-12-15T01:18:49.864ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
5602023-12-15T01:18:49.864ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 1433420e-118d-4f4e-b816-04dd2f29c21e dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
5612023-12-15T01:18:49.864ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
5622023-12-15T01:18:49.865ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
5632023-12-15T01:18:49.865ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
5642023-12-15T01:18:49.865ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
5652023-12-15T01:18:49.865ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
5662023-12-15T01:18:49.865ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
5672023-12-15T01:18:49.865ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
5682023-12-15T01:18:49.865ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
5692023-12-15T01:18:49.866ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5702023-12-15T01:18:49.866ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
5712023-12-15T01:18:49.867ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
5722023-12-15T01:18:49.867ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = b55d5eea-f93a-43bf-9796-c715ecea5842 dns_zone_name = oxide.test found_so_far = 0 version = 2
5732023-12-15T01:18:49.867ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
5742023-12-15T01:18:49.867ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
5752023-12-15T01:18:49.867ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
5762023-12-15T01:18:49.868ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5772023-12-15T01:18:49.868ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5782023-12-15T01:18:49.868ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
5792023-12-15T01:18:49.868ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
5802023-12-15T01:18:49.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
5812023-12-15T01:18:49.870ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 1433420e-118d-4f4e-b816-04dd2f29c21e dns_zone_name = control-plane.oxide.internal found_so_far = 51 version = 1
5822023-12-15T01:18:49.870ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
5832023-12-15T01:18:49.870ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2023-12-15 01:14:03.961426 UTC dns_group = internal generation = 1
5842023-12-15T01:18:49.870ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 20.996136ms iteration = 7
5852023-12-15T01:18:49.871ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
5862023-12-15T01:18:49.871ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
5872023-12-15T01:18:49.871ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
5882023-12-15T01:18:49.872ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
5892023-12-15T01:18:49.872ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
5902023-12-15T01:18:49.872ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
5912023-12-15T01:18:49.873ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = b55d5eea-f93a-43bf-9796-c715ecea5842 dns_zone_name = oxide.test found_so_far = 1 version = 2
5922023-12-15T01:18:49.873ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
5932023-12-15T01:18:49.873ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2023-12-15 01:14:04.383795 UTC dns_group = external generation = 2
5942023-12-15T01:18:49.873ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 21.490184ms iteration = 7
5952023-12-15T01:18:49.881ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2023-12-15T01:13:49.851170Z, time_modified: 2023-12-15T01:13:49.851170Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: bcd56901-6236-45d5-99fd-58ae6b425f02, db_silo: Silo { identity: SiloIdentity { id: bcd56901-6236-45d5-99fd-58ae6b425f02, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2023-12-15T01:14:04.327942Z, time_modified: 2023-12-15T01:14:04.327942Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "6a0b318c733ecc8fe690fee9446b5a046a1e6779347663bbc3588532a4ce10cc" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: bcd56901-6236-45d5-99fd-58ae6b425f02, db_silo: Silo { identity: SiloIdentity { id: bcd56901-6236-45d5-99fd-58ae6b425f02, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2023-12-15T01:14:04.327942Z, time_modified: 2023-12-15T01:14:04.327942Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "6a0b318c733ecc8fe690fee9446b5a046a1e6779347663bbc3588532a4ce10cc" }] }) }
5962023-12-15T01:18:49.881ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 29.732977ms iteration = 7
5972023-12-15T01:18:49.920ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
5982023-12-15T01:18:49.923ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
5992023-12-15T01:18:49.978ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
6002023-12-15T01:18:49.979ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::8)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "5415345e-3c55-4a22-86dd-62db9350d8e0", "date": "Fri, 15 Dec 2023 01:18:49 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
6012023-12-15T01:18:49.979ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/dns_propagation.rs:173 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
6022023-12-15T01:18:49.980ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
6032023-12-15T01:18:49.981ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::9)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "01482346-db01-4c79-9f9b-8374ce7e2079", "date": "Fri, 15 Dec 2023 01:18:49 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
6042023-12-15T01:18:49.981ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/dns_propagation.rs:173 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
6052023-12-15T01:18:49.982ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/dns_propagation.rs:110 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
6062023-12-15T01:18:49.982ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 130.547236ms iteration = 9
6072023-12-15T01:18:50.010ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
6082023-12-15T01:18:50.011ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:3::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "ae7987b7-3b17-4e27-93ba-dbe9f107a1af", "date": "Fri, 15 Dec 2023 01:18:49 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
6092023-12-15T01:18:50.011ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/dns_propagation.rs:173 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
6102023-12-15T01:18:50.013ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:1::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "97c1862d-7ba6-4d8f-92cd-cec826dfc82d", "date": "Fri, 15 Dec 2023 01:18:49 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
6112023-12-15T01:18:50.013ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/dns_propagation.rs:173 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
6122023-12-15T01:18:50.014ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:2::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "a7086359-2905-4202-b014-bae0b782e56d", "date": "Fri, 15 Dec 2023 01:18:49 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
6132023-12-15T01:18:50.014ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/dns_propagation.rs:173 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
6142023-12-15T01:18:50.015ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/dns_propagation.rs:110 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
6152023-12-15T01:18:50.015ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 163.678664ms iteration = 9
6162023-12-15T01:18:56.144ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 1289 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:60064 req_id = f4a85db2-a3f9-4d24-9ced-71b60b078078 response_code = 200 uri = /metrics/collect/700e7b49-f840-40d5-92e8-83ab758ea8a7
6172023-12-15T01:19:06.144ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 1130 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:60064 req_id = 4c992bae-8023-4618-a9c6-3ce94d424127 response_code = 200 uri = /metrics/collect/700e7b49-f840-40d5-92e8-83ab758ea8a7
6182023-12-15T01:19:16.143ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 608 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:60064 req_id = 9b1ebd7f-8d57-4a93-a976-f84c7a60ec35 response_code = 200 uri = /metrics/collect/700e7b49-f840-40d5-92e8-83ab758ea8a7
6192023-12-15T01:19:19.786ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:46101 req_id = a17fa6b6-21e3-4be0-b728-b738ba7bf0a8 roles = RoleSet { roles: {} } uri = /nat/ipv4/changeset/0?limit=100
6202023-12-15T01:19:19.786ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:46101 req_id = a17fa6b6-21e3-4be0-b728-b738ba7bf0a8 resource = Database result = Ok(()) uri = /nat/ipv4/changeset/0?limit=100
6212023-12-15T01:19:19.788ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 2654 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:46101 req_id = a17fa6b6-21e3-4be0-b728-b738ba7bf0a8 response_code = 200 uri = /nat/ipv4/changeset/0?limit=100
6222023-12-15T01:19:19.854ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 12 reason = Timeout
6232023-12-15T01:19:19.854ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 12 reason = Timeout
6242023-12-15T01:19:19.854ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
6252023-12-15T01:19:19.854ZWARNnexus (BackgroundTasks): phantom disk task started background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:50
6262023-12-15T01:19:19.854ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
6272023-12-15T01:19:19.856ZDEBGnexus (DpdClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
6282023-12-15T01:19:19.856ZDEBGnexus (DpdClient): client response result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "33fa5541-0210-468b-b47b-9bb6eb46bbc1", "content-length": "1", "date": "Fri, 15 Dec 2023 01:19:18 GMT"} })
6292023-12-15T01:19:19.856ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
6302023-12-15T01:19:19.857ZWARNnexus (BackgroundTasks): phantom disk task done background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:96
6312023-12-15T01:19:19.857ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 2.549238ms iteration = 12
6322023-12-15T01:19:19.857ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
6332023-12-15T01:19:19.858ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 3.972879ms iteration = 12
6342023-12-15T01:19:26.143ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 719 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:60064 req_id = d3f1f032-dd1e-462e-a075-6cb7c6deecf7 response_code = 200 uri = /metrics/collect/700e7b49-f840-40d5-92e8-83ab758ea8a7
6352023-12-15T01:19:36.144ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 1161 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:60064 req_id = 8f94594b-3d66-4e4a-8f5c-c0440c48dfb5 response_code = 200 uri = /metrics/collect/700e7b49-f840-40d5-92e8-83ab758ea8a7
6362023-12-15T01:19:46.143ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 717 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:60064 req_id = e099ce88-e5a5-4caf-aeb5-2b0948e27ce9 response_code = 200 uri = /metrics/collect/700e7b49-f840-40d5-92e8-83ab758ea8a7
6372023-12-15T01:19:49.818ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:46101 req_id = f3dfb4b6-45fc-49c5-bfd1-a3d4b55c348c roles = RoleSet { roles: {} } uri = /nat/ipv4/changeset/0?limit=100
6382023-12-15T01:19:49.818ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:46101 req_id = f3dfb4b6-45fc-49c5-bfd1-a3d4b55c348c resource = Database result = Ok(()) uri = /nat/ipv4/changeset/0?limit=100
6392023-12-15T01:19:49.821ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 2909 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:46101 req_id = f3dfb4b6-45fc-49c5-bfd1-a3d4b55c348c response_code = 200 uri = /nat/ipv4/changeset/0?limit=100
6402023-12-15T01:19:49.850ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 8 reason = Timeout
6412023-12-15T01:19:49.850ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
6422023-12-15T01:19:49.850ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
6432023-12-15T01:19:49.850ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
6442023-12-15T01:19:49.851ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
6452023-12-15T01:19:49.851ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
6462023-12-15T01:19:49.851ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 10 reason = Timeout
6472023-12-15T01:19:49.851ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 8 reason = Timeout
6482023-12-15T01:19:49.851ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
6492023-12-15T01:19:49.851ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
6502023-12-15T01:19:49.851ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 8 reason = Timeout
6512023-12-15T01:19:49.851ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 8 reason = Timeout
6522023-12-15T01:19:49.851ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_external dns_group = external roles = RoleSet { roles: {} }
6532023-12-15T01:19:49.851ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 8 reason = Timeout
6542023-12-15T01:19:49.851ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
6552023-12-15T01:19:49.851ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 10 reason = Timeout
6562023-12-15T01:19:49.851ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_internal dns_group = internal roles = RoleSet { roles: {} }
6572023-12-15T01:19:49.851ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
6582023-12-15T01:19:49.851ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_external dns_group = external resource = Database result = Ok(())
6592023-12-15T01:19:49.852ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
6602023-12-15T01:19:49.852ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_internal dns_group = internal resource = Database result = Ok(())
6612023-12-15T01:19:49.852ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
6622023-12-15T01:19:49.852ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
6632023-12-15T01:19:49.852ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6642023-12-15T01:19:49.853ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6652023-12-15T01:19:49.853ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 13 reason = Timeout
6662023-12-15T01:19:49.853ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
6672023-12-15T01:19:49.853ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 13 reason = Timeout
6682023-12-15T01:19:49.853ZWARNnexus (BackgroundTasks): phantom disk task started background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:50
6692023-12-15T01:19:49.853ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6702023-12-15T01:19:49.853ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
6712023-12-15T01:19:49.854ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6722023-12-15T01:19:49.854ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6732023-12-15T01:19:49.855ZDEBGnexus (DpdClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
6742023-12-15T01:19:49.855ZDEBGnexus (DpdClient): client response result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "343f86c4-8274-4694-83a8-9c84d4bfa31e", "content-length": "1", "date": "Fri, 15 Dec 2023 01:19:49 GMT"} })
6752023-12-15T01:19:49.855ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
6762023-12-15T01:19:49.855ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
6772023-12-15T01:19:49.856ZWARNnexus (BackgroundTasks): phantom disk task done background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:96
6782023-12-15T01:19:49.856ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 2.543699ms iteration = 13
6792023-12-15T01:19:49.856ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
6802023-12-15T01:19:49.857ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 3.826577ms iteration = 13
6812023-12-15T01:19:49.857ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
6822023-12-15T01:19:49.857ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
6832023-12-15T01:19:49.857ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
6842023-12-15T01:19:49.857ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
6852023-12-15T01:19:49.857ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
6862023-12-15T01:19:49.858ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
6872023-12-15T01:19:49.858ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
6882023-12-15T01:19:49.858ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_internal dns_group = internal resource = Fleet result = Ok(())
6892023-12-15T01:19:49.858ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_internal dns_group = internal roles = RoleSet { roles: {} }
6902023-12-15T01:19:49.858ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
6912023-12-15T01:19:49.858ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_internal dns_group = internal resource = Database result = Ok(())
6922023-12-15T01:19:49.858ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6932023-12-15T01:19:49.859ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_external dns_group = external resource = Fleet result = Ok(())
6942023-12-15T01:19:49.859ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_external dns_group = external roles = RoleSet { roles: {} }
6952023-12-15T01:19:49.859ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
6962023-12-15T01:19:49.859ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_external dns_group = external resource = Database result = Ok(())
6972023-12-15T01:19:49.860ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
6982023-12-15T01:19:49.860ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
6992023-12-15T01:19:49.860ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
7002023-12-15T01:19:49.860ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
7012023-12-15T01:19:49.860ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
7022023-12-15T01:19:49.860ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
7032023-12-15T01:19:49.860ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
7042023-12-15T01:19:49.860ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } dns_group = internal
7052023-12-15T01:19:49.860ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 9.216719ms iteration = 8
7062023-12-15T01:19:49.861ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } dns_group = external
7072023-12-15T01:19:49.861ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 9.487623ms iteration = 8
7082023-12-15T01:19:49.861ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7092023-12-15T01:19:49.861ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7102023-12-15T01:19:49.861ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
7112023-12-15T01:19:49.861ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
7122023-12-15T01:19:49.862ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
7132023-12-15T01:19:49.863ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
7142023-12-15T01:19:49.863ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 1433420e-118d-4f4e-b816-04dd2f29c21e dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
7152023-12-15T01:19:49.863ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
7162023-12-15T01:19:49.863ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
7172023-12-15T01:19:49.864ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7182023-12-15T01:19:49.864ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
7192023-12-15T01:19:49.864ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
7202023-12-15T01:19:49.864ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
7212023-12-15T01:19:49.865ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
7222023-12-15T01:19:49.865ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
7232023-12-15T01:19:49.865ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
7242023-12-15T01:19:49.866ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
7252023-12-15T01:19:49.866ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = b55d5eea-f93a-43bf-9796-c715ecea5842 dns_zone_name = oxide.test found_so_far = 0 version = 2
7262023-12-15T01:19:49.866ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
7272023-12-15T01:19:49.866ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
7282023-12-15T01:19:49.867ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
7292023-12-15T01:19:49.867ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
7302023-12-15T01:19:49.867ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
7312023-12-15T01:19:49.867ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
7322023-12-15T01:19:49.867ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7332023-12-15T01:19:49.867ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
7342023-12-15T01:19:49.868ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7352023-12-15T01:19:49.869ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 1433420e-118d-4f4e-b816-04dd2f29c21e dns_zone_name = control-plane.oxide.internal found_so_far = 51 version = 1
7362023-12-15T01:19:49.869ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
7372023-12-15T01:19:49.869ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2023-12-15 01:14:03.961426 UTC dns_group = internal generation = 1
7382023-12-15T01:19:49.869ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 18.989946ms iteration = 8
7392023-12-15T01:19:49.870ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
7402023-12-15T01:19:49.870ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
7412023-12-15T01:19:49.870ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
7422023-12-15T01:19:49.870ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
7432023-12-15T01:19:49.870ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
7442023-12-15T01:19:49.871ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
7452023-12-15T01:19:49.872ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = b55d5eea-f93a-43bf-9796-c715ecea5842 dns_zone_name = oxide.test found_so_far = 1 version = 2
7462023-12-15T01:19:49.872ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
7472023-12-15T01:19:49.872ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2023-12-15 01:14:04.383795 UTC dns_group = external generation = 2
7482023-12-15T01:19:49.872ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 20.695792ms iteration = 8
7492023-12-15T01:19:49.876ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2023-12-15T01:13:49.851170Z, time_modified: 2023-12-15T01:13:49.851170Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: bcd56901-6236-45d5-99fd-58ae6b425f02, db_silo: Silo { identity: SiloIdentity { id: bcd56901-6236-45d5-99fd-58ae6b425f02, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2023-12-15T01:14:04.327942Z, time_modified: 2023-12-15T01:14:04.327942Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "6a0b318c733ecc8fe690fee9446b5a046a1e6779347663bbc3588532a4ce10cc" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: bcd56901-6236-45d5-99fd-58ae6b425f02, db_silo: Silo { identity: SiloIdentity { id: bcd56901-6236-45d5-99fd-58ae6b425f02, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2023-12-15T01:14:04.327942Z, time_modified: 2023-12-15T01:14:04.327942Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "6a0b318c733ecc8fe690fee9446b5a046a1e6779347663bbc3588532a4ce10cc" }] }) }
7502023-12-15T01:19:49.876ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 24.814784ms iteration = 8
7512023-12-15T01:19:49.892ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
7522023-12-15T01:19:49.892ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
7532023-12-15T01:19:49.929ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
7542023-12-15T01:19:49.930ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
7552023-12-15T01:19:49.930ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::9)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "a9f68449-eb8c-4922-80e4-36eda5eacc75", "date": "Fri, 15 Dec 2023 01:19:49 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
7562023-12-15T01:19:49.930ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/dns_propagation.rs:173 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
7572023-12-15T01:19:49.931ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::8)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "d6caa378-5ea4-44da-89d1-81bc057e21c5", "date": "Fri, 15 Dec 2023 01:19:49 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
7582023-12-15T01:19:49.931ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/dns_propagation.rs:173 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
7592023-12-15T01:19:49.931ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/dns_propagation.rs:110 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
7602023-12-15T01:19:49.931ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 79.993185ms iteration = 10
7612023-12-15T01:19:49.959ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
7622023-12-15T01:19:49.959ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:1::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "dc7abe19-1e7a-4e97-90aa-c590cc4264d5", "date": "Fri, 15 Dec 2023 01:19:49 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
7632023-12-15T01:19:49.959ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/dns_propagation.rs:173 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
7642023-12-15T01:19:49.961ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:2::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "32966b4f-d744-4663-b50e-531f3bbbbbca", "date": "Fri, 15 Dec 2023 01:19:49 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
7652023-12-15T01:19:49.961ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/dns_propagation.rs:173 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
7662023-12-15T01:19:49.961ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:3::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "c2fe4577-2928-4d1d-93ae-2055abd17506", "date": "Fri, 15 Dec 2023 01:19:49 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
7672023-12-15T01:19:49.961ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/dns_propagation.rs:173 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
7682023-12-15T01:19:49.962ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/dns_propagation.rs:110 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
7692023-12-15T01:19:49.962ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 111.059142ms iteration = 10
7702023-12-15T01:19:56.143ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 714 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:60064 req_id = bfcafa65-b241-420c-9f04-9f7e7fccc13c response_code = 200 uri = /metrics/collect/700e7b49-f840-40d5-92e8-83ab758ea8a7
7712023-12-15T01:20:06.144ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 1149 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:60064 req_id = 986f8a23-c4a8-4a46-9ca8-2855b75fbb34 response_code = 200 uri = /metrics/collect/700e7b49-f840-40d5-92e8-83ab758ea8a7
7722023-12-15T01:20:16.143ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 831 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:60064 req_id = fef7ae2d-6c82-48ed-a150-226686a1efad response_code = 200 uri = /metrics/collect/700e7b49-f840-40d5-92e8-83ab758ea8a7
7732023-12-15T01:20:19.851ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:46101 req_id = fe138eea-4d36-47b3-a2bb-5b9c0c2d3b3e roles = RoleSet { roles: {} } uri = /nat/ipv4/changeset/0?limit=100
7742023-12-15T01:20:19.852ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:46101 req_id = fe138eea-4d36-47b3-a2bb-5b9c0c2d3b3e resource = Database result = Ok(()) uri = /nat/ipv4/changeset/0?limit=100
7752023-12-15T01:20:19.853ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 14 reason = Timeout
7762023-12-15T01:20:19.853ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 14 reason = Timeout
7772023-12-15T01:20:19.853ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
7782023-12-15T01:20:19.853ZWARNnexus (BackgroundTasks): phantom disk task started background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:50
7792023-12-15T01:20:19.853ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
7802023-12-15T01:20:19.854ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 2293 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:46101 req_id = fe138eea-4d36-47b3-a2bb-5b9c0c2d3b3e response_code = 200 uri = /nat/ipv4/changeset/0?limit=100
7812023-12-15T01:20:19.855ZDEBGnexus (DpdClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
7822023-12-15T01:20:19.855ZDEBGnexus (DpdClient): client response result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "6cbdf4ca-1700-4e2b-8b51-403c35873a64", "content-length": "1", "date": "Fri, 15 Dec 2023 01:20:19 GMT"} })
7832023-12-15T01:20:19.855ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
7842023-12-15T01:20:19.855ZWARNnexus (BackgroundTasks): phantom disk task done background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:96
7852023-12-15T01:20:19.855ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 2.327245ms iteration = 14
7862023-12-15T01:20:19.856ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
7872023-12-15T01:20:19.857ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 3.452672ms iteration = 14
7882023-12-15T01:20:26.143ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 1008 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:60064 req_id = 331f405c-7ab6-4d80-9a35-545449c3c333 response_code = 200 uri = /metrics/collect/700e7b49-f840-40d5-92e8-83ab758ea8a7
7892023-12-15T01:20:29.864ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:46101 req_id = 3856a221-c588-49e9-8123-80be395d5f6b roles = RoleSet { roles: {} } uri = /nat/ipv4/changeset/0?limit=100
7902023-12-15T01:20:29.864ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:46101 req_id = 3856a221-c588-49e9-8123-80be395d5f6b resource = Database result = Ok(()) uri = /nat/ipv4/changeset/0?limit=100
7912023-12-15T01:20:29.867ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 3331 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:46101 req_id = 3856a221-c588-49e9-8123-80be395d5f6b response_code = 200 uri = /nat/ipv4/changeset/0?limit=100
7922023-12-15T01:20:29.872ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:46101 req_id = 69820f77-0704-4a11-9798-43906d11ef47 roles = RoleSet { roles: {} } uri = /nat/ipv4/changeset/2?limit=100
7932023-12-15T01:20:29.872ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:46101 req_id = 69820f77-0704-4a11-9798-43906d11ef47 resource = Database result = Ok(()) uri = /nat/ipv4/changeset/2?limit=100
7942023-12-15T01:20:29.874ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 2422 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:46101 req_id = 69820f77-0704-4a11-9798-43906d11ef47 response_code = 200 uri = /nat/ipv4/changeset/2?limit=100
7952023-12-15T01:20:36.142ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 656 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:60064 req_id = 3ca8fbda-22ed-493a-8bef-87359c349094 response_code = 200 uri = /metrics/collect/700e7b49-f840-40d5-92e8-83ab758ea8a7
7962023-12-15T01:20:37.023ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:769 local_addr = [fd00:1122:3344:101::c]:12221 remote_addr = [fd00:1122:3344:101::1]:47834
7972023-12-15T01:20:37.027ZINFOnexus: received new runtime state from sled agent file = nexus/src/app/instance.rs:1392 instance_id = 4aa645b6-9ac6-43e1-afd5-dec8793ba69d instance_state = InstanceRuntimeState { propolis_id: Some(7bfbb4c7-9544-4c62-b705-404d31141456), dst_propolis_id: None, migration_id: None, gen: Generation(3), time_updated: 2023-12-15T01:20:29.087700138Z } propolis_id = 7bfbb4c7-9544-4c62-b705-404d31141456 vmm_state = VmmRuntimeState { state: Starting, gen: Generation(2), time_updated: 2023-12-15T01:20:37.021155857Z }
7982023-12-15T01:20:37.030ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:47834 req_id = baeeb7e3-2f81-43dc-96eb-ebe1c752f5c4 roles = RoleSet { roles: {} } uri = /instances/4aa645b6-9ac6-43e1-afd5-dec8793ba69d
7992023-12-15T01:20:37.031ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:47834 req_id = baeeb7e3-2f81-43dc-96eb-ebe1c752f5c4 resource = Database result = Ok(()) uri = /instances/4aa645b6-9ac6-43e1-afd5-dec8793ba69d
8002023-12-15T01:20:37.038ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:47834 req_id = baeeb7e3-2f81-43dc-96eb-ebe1c752f5c4 roles = RoleSet { roles: {} } uri = /instances/4aa645b6-9ac6-43e1-afd5-dec8793ba69d
8012023-12-15T01:20:37.038ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:47834 req_id = baeeb7e3-2f81-43dc-96eb-ebe1c752f5c4 resource = Database result = Ok(()) uri = /instances/4aa645b6-9ac6-43e1-afd5-dec8793ba69d
8022023-12-15T01:20:37.039ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:47834 req_id = baeeb7e3-2f81-43dc-96eb-ebe1c752f5c4 roles = RoleSet { roles: {} } uri = /instances/4aa645b6-9ac6-43e1-afd5-dec8793ba69d
8032023-12-15T01:20:37.040ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:47834 req_id = baeeb7e3-2f81-43dc-96eb-ebe1c752f5c4 resource = Database result = Ok(()) uri = /instances/4aa645b6-9ac6-43e1-afd5-dec8793ba69d
8042023-12-15T01:20:37.041ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:47834 req_id = baeeb7e3-2f81-43dc-96eb-ebe1c752f5c4 roles = RoleSet { roles: {} } uri = /instances/4aa645b6-9ac6-43e1-afd5-dec8793ba69d
8052023-12-15T01:20:37.041ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:47834 req_id = baeeb7e3-2f81-43dc-96eb-ebe1c752f5c4 resource = Database result = Ok(()) uri = /instances/4aa645b6-9ac6-43e1-afd5-dec8793ba69d
8062023-12-15T01:20:37.043ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:47834 req_id = baeeb7e3-2f81-43dc-96eb-ebe1c752f5c4 roles = RoleSet { roles: {} } uri = /instances/4aa645b6-9ac6-43e1-afd5-dec8793ba69d
8072023-12-15T01:20:37.043ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:47834 req_id = baeeb7e3-2f81-43dc-96eb-ebe1c752f5c4 resource = Database result = Ok(()) uri = /instances/4aa645b6-9ac6-43e1-afd5-dec8793ba69d
8082023-12-15T01:20:37.044ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:47834 req_id = baeeb7e3-2f81-43dc-96eb-ebe1c752f5c4 roles = RoleSet { roles: {} } uri = /instances/4aa645b6-9ac6-43e1-afd5-dec8793ba69d
8092023-12-15T01:20:37.044ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:47834 req_id = baeeb7e3-2f81-43dc-96eb-ebe1c752f5c4 resource = Database result = Ok(()) uri = /instances/4aa645b6-9ac6-43e1-afd5-dec8793ba69d
8102023-12-15T01:20:37.046ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:47834 req_id = baeeb7e3-2f81-43dc-96eb-ebe1c752f5c4 roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } uri = /instances/4aa645b6-9ac6-43e1-afd5-dec8793ba69d
8112023-12-15T01:20:37.050ZDEBGnexus (dropshot_internal): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:47834 req_id = baeeb7e3-2f81-43dc-96eb-ebe1c752f5c4 resource = Instance { parent: Project { parent: Silo { parent: Fleet, key: bcd56901-6236-45d5-99fd-58ae6b425f02, lookup_type: ById(bcd56901-6236-45d5-99fd-58ae6b425f02) }, key: 5cde8df0-a9e4-46c4-825b-920baa50c1ca, lookup_type: ById(5cde8df0-a9e4-46c4-825b-920baa50c1ca) }, key: 4aa645b6-9ac6-43e1-afd5-dec8793ba69d, lookup_type: ById(4aa645b6-9ac6-43e1-afd5-dec8793ba69d) } result = Ok(()) uri = /instances/4aa645b6-9ac6-43e1-afd5-dec8793ba69d
8122023-12-15T01:20:37.050ZDEBGnexus: instance state generation already advanced, won't touch network config instance_id = 4aa645b6-9ac6-43e1-afd5-dec8793ba69d
8132023-12-15T01:20:37.100ZINFOnexus: instance and vmm updated by sled agent file = nexus/src/app/instance.rs:1528 instance_id = 4aa645b6-9ac6-43e1-afd5-dec8793ba69d instance_updated = false propolis_id = 7bfbb4c7-9544-4c62-b705-404d31141456 vmm_updated = true
8142023-12-15T01:20:37.100ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 76574 local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:47834 req_id = baeeb7e3-2f81-43dc-96eb-ebe1c752f5c4 response_code = 204 uri = /instances/4aa645b6-9ac6-43e1-afd5-dec8793ba69d
8152023-12-15T01:20:37.101ZINFOnexus: received new runtime state from sled agent file = nexus/src/app/instance.rs:1392 instance_id = 4aa645b6-9ac6-43e1-afd5-dec8793ba69d instance_state = InstanceRuntimeState { propolis_id: Some(7bfbb4c7-9544-4c62-b705-404d31141456), dst_propolis_id: None, migration_id: None, gen: Generation(3), time_updated: 2023-12-15T01:20:29.087700138Z } propolis_id = 7bfbb4c7-9544-4c62-b705-404d31141456 vmm_state = VmmRuntimeState { state: Running, gen: Generation(3), time_updated: 2023-12-15T01:20:37.100973639Z }
8162023-12-15T01:20:37.101ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:47834 req_id = 50e0879f-09d0-428b-bbda-6ee5d7d7c21e roles = RoleSet { roles: {} } uri = /instances/4aa645b6-9ac6-43e1-afd5-dec8793ba69d
8172023-12-15T01:20:37.101ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:47834 req_id = 50e0879f-09d0-428b-bbda-6ee5d7d7c21e resource = Database result = Ok(()) uri = /instances/4aa645b6-9ac6-43e1-afd5-dec8793ba69d
8182023-12-15T01:20:37.102ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:47834 req_id = 50e0879f-09d0-428b-bbda-6ee5d7d7c21e roles = RoleSet { roles: {} } uri = /instances/4aa645b6-9ac6-43e1-afd5-dec8793ba69d
8192023-12-15T01:20:37.103ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:47834 req_id = 50e0879f-09d0-428b-bbda-6ee5d7d7c21e resource = Database result = Ok(()) uri = /instances/4aa645b6-9ac6-43e1-afd5-dec8793ba69d
8202023-12-15T01:20:37.104ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:47834 req_id = 50e0879f-09d0-428b-bbda-6ee5d7d7c21e roles = RoleSet { roles: {} } uri = /instances/4aa645b6-9ac6-43e1-afd5-dec8793ba69d
8212023-12-15T01:20:37.104ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:47834 req_id = 50e0879f-09d0-428b-bbda-6ee5d7d7c21e resource = Database result = Ok(()) uri = /instances/4aa645b6-9ac6-43e1-afd5-dec8793ba69d
8222023-12-15T01:20:37.105ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:47834 req_id = 50e0879f-09d0-428b-bbda-6ee5d7d7c21e roles = RoleSet { roles: {} } uri = /instances/4aa645b6-9ac6-43e1-afd5-dec8793ba69d
8232023-12-15T01:20:37.105ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:47834 req_id = 50e0879f-09d0-428b-bbda-6ee5d7d7c21e resource = Database result = Ok(()) uri = /instances/4aa645b6-9ac6-43e1-afd5-dec8793ba69d
8242023-12-15T01:20:37.107ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:47834 req_id = 50e0879f-09d0-428b-bbda-6ee5d7d7c21e roles = RoleSet { roles: {} } uri = /instances/4aa645b6-9ac6-43e1-afd5-dec8793ba69d
8252023-12-15T01:20:37.107ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:47834 req_id = 50e0879f-09d0-428b-bbda-6ee5d7d7c21e resource = Database result = Ok(()) uri = /instances/4aa645b6-9ac6-43e1-afd5-dec8793ba69d
8262023-12-15T01:20:37.108ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:47834 req_id = 50e0879f-09d0-428b-bbda-6ee5d7d7c21e roles = RoleSet { roles: {} } uri = /instances/4aa645b6-9ac6-43e1-afd5-dec8793ba69d
8272023-12-15T01:20:37.108ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:47834 req_id = 50e0879f-09d0-428b-bbda-6ee5d7d7c21e resource = Database result = Ok(()) uri = /instances/4aa645b6-9ac6-43e1-afd5-dec8793ba69d
8282023-12-15T01:20:37.109ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:47834 req_id = 50e0879f-09d0-428b-bbda-6ee5d7d7c21e roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } uri = /instances/4aa645b6-9ac6-43e1-afd5-dec8793ba69d
8292023-12-15T01:20:37.112ZDEBGnexus (dropshot_internal): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:47834 req_id = 50e0879f-09d0-428b-bbda-6ee5d7d7c21e resource = Instance { parent: Project { parent: Silo { parent: Fleet, key: bcd56901-6236-45d5-99fd-58ae6b425f02, lookup_type: ById(bcd56901-6236-45d5-99fd-58ae6b425f02) }, key: 5cde8df0-a9e4-46c4-825b-920baa50c1ca, lookup_type: ById(5cde8df0-a9e4-46c4-825b-920baa50c1ca) }, key: 4aa645b6-9ac6-43e1-afd5-dec8793ba69d, lookup_type: ById(4aa645b6-9ac6-43e1-afd5-dec8793ba69d) } result = Ok(()) uri = /instances/4aa645b6-9ac6-43e1-afd5-dec8793ba69d
8302023-12-15T01:20:37.112ZDEBGnexus: instance state generation already advanced, won't touch network config instance_id = 4aa645b6-9ac6-43e1-afd5-dec8793ba69d
8312023-12-15T01:20:37.150ZINFOnexus: instance and vmm updated by sled agent file = nexus/src/app/instance.rs:1528 instance_id = 4aa645b6-9ac6-43e1-afd5-dec8793ba69d instance_updated = false propolis_id = 7bfbb4c7-9544-4c62-b705-404d31141456 vmm_updated = true
8322023-12-15T01:20:37.150ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 48959 local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:47834 req_id = 50e0879f-09d0-428b-bbda-6ee5d7d7c21e response_code = 204 uri = /instances/4aa645b6-9ac6-43e1-afd5-dec8793ba69d
8332023-12-15T01:20:46.143ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 775 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:60064 req_id = d574c271-4c81-46b0-a002-c393ed4f86b2 response_code = 200 uri = /metrics/collect/700e7b49-f840-40d5-92e8-83ab758ea8a7
8342023-12-15T01:20:49.849ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 9 reason = Timeout
8352023-12-15T01:20:49.849ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
8362023-12-15T01:20:49.849ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
8372023-12-15T01:20:49.849ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
8382023-12-15T01:20:49.850ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
8392023-12-15T01:20:49.850ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
8402023-12-15T01:20:49.851ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8412023-12-15T01:20:49.851ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 11 reason = Timeout
8422023-12-15T01:20:49.851ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 9 reason = Timeout
8432023-12-15T01:20:49.851ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
8442023-12-15T01:20:49.851ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 9 reason = Timeout
8452023-12-15T01:20:49.851ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
8462023-12-15T01:20:49.851ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_internal dns_group = internal roles = RoleSet { roles: {} }
8472023-12-15T01:20:49.851ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 9 reason = Timeout
8482023-12-15T01:20:49.851ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 9 reason = Timeout
8492023-12-15T01:20:49.851ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_external dns_group = external roles = RoleSet { roles: {} }
8502023-12-15T01:20:49.851ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 11 reason = Timeout
8512023-12-15T01:20:49.851ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
8522023-12-15T01:20:49.852ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
8532023-12-15T01:20:49.852ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_internal dns_group = internal resource = Database result = Ok(())
8542023-12-15T01:20:49.852ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
8552023-12-15T01:20:49.852ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_external dns_group = external resource = Database result = Ok(())
8562023-12-15T01:20:49.852ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
8572023-12-15T01:20:49.852ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
8582023-12-15T01:20:49.853ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8592023-12-15T01:20:49.853ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8602023-12-15T01:20:49.853ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8612023-12-15T01:20:49.853ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 15 reason = Timeout
8622023-12-15T01:20:49.853ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 15 reason = Timeout
8632023-12-15T01:20:49.853ZWARNnexus (BackgroundTasks): phantom disk task started background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:50
8642023-12-15T01:20:49.853ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
8652023-12-15T01:20:49.853ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8662023-12-15T01:20:49.854ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
8672023-12-15T01:20:49.855ZDEBGnexus (DpdClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
8682023-12-15T01:20:49.855ZDEBGnexus (DpdClient): client response result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "c845fdac-87a6-4050-9867-a83717461e0d", "content-length": "1", "date": "Fri, 15 Dec 2023 01:20:49 GMT"} })
8692023-12-15T01:20:49.855ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
8702023-12-15T01:20:49.855ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
8712023-12-15T01:20:49.856ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
8722023-12-15T01:20:49.856ZWARNnexus (BackgroundTasks): phantom disk task done background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:96
8732023-12-15T01:20:49.856ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 2.763752ms iteration = 15
8742023-12-15T01:20:49.856ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
8752023-12-15T01:20:49.856ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
8762023-12-15T01:20:49.856ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
8772023-12-15T01:20:49.856ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
8782023-12-15T01:20:49.857ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 3.34141ms iteration = 15
8792023-12-15T01:20:49.857ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
8802023-12-15T01:20:49.857ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_internal dns_group = internal resource = Fleet result = Ok(())
8812023-12-15T01:20:49.857ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_internal dns_group = internal roles = RoleSet { roles: {} }
8822023-12-15T01:20:49.857ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_internal dns_group = internal resource = Database result = Ok(())
8832023-12-15T01:20:49.858ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_external dns_group = external resource = Fleet result = Ok(())
8842023-12-15T01:20:49.858ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_external dns_group = external roles = RoleSet { roles: {} }
8852023-12-15T01:20:49.858ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_external dns_group = external resource = Database result = Ok(())
8862023-12-15T01:20:49.858ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8872023-12-15T01:20:49.858ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
8882023-12-15T01:20:49.858ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
8892023-12-15T01:20:49.858ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
8902023-12-15T01:20:49.858ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
8912023-12-15T01:20:49.859ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
8922023-12-15T01:20:49.859ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
8932023-12-15T01:20:49.859ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
8942023-12-15T01:20:49.859ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
8952023-12-15T01:20:49.859ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
8962023-12-15T01:20:49.859ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } dns_group = external
8972023-12-15T01:20:49.859ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 8.174903ms iteration = 9
8982023-12-15T01:20:49.859ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
8992023-12-15T01:20:49.859ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
9002023-12-15T01:20:49.860ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9012023-12-15T01:20:49.860ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
9022023-12-15T01:20:49.860ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
9032023-12-15T01:20:49.861ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9042023-12-15T01:20:49.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
9052023-12-15T01:20:49.864ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
9062023-12-15T01:20:49.864ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
9072023-12-15T01:20:49.864ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
9082023-12-15T01:20:49.864ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
9092023-12-15T01:20:49.864ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
9102023-12-15T01:20:49.864ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
9112023-12-15T01:20:49.865ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
9122023-12-15T01:20:49.866ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
9132023-12-15T01:20:49.866ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = b55d5eea-f93a-43bf-9796-c715ecea5842 dns_zone_name = oxide.test found_so_far = 0 version = 2
9142023-12-15T01:20:49.866ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
9152023-12-15T01:20:49.866ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
9162023-12-15T01:20:49.866ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
9172023-12-15T01:20:49.867ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9182023-12-15T01:20:49.867ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9192023-12-15T01:20:49.869ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
9202023-12-15T01:20:49.869ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
9212023-12-15T01:20:49.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
9222023-12-15T01:20:49.870ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
9232023-12-15T01:20:49.870ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
9242023-12-15T01:20:49.870ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
9252023-12-15T01:20:49.871ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = b55d5eea-f93a-43bf-9796-c715ecea5842 dns_zone_name = oxide.test found_so_far = 1 version = 2
9262023-12-15T01:20:49.871ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
9272023-12-15T01:20:49.871ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2023-12-15 01:14:04.383795 UTC dns_group = external generation = 2
9282023-12-15T01:20:49.871ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 20.089653ms iteration = 9
9292023-12-15T01:20:49.874ZINFOnexus (BackgroundTasks): found Silo/DNS/TLS config (changed) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2023-12-15T01:13:49.851170Z, time_modified: 2023-12-15T01:13:49.851170Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: bcd56901-6236-45d5-99fd-58ae6b425f02, db_silo: Silo { identity: SiloIdentity { id: bcd56901-6236-45d5-99fd-58ae6b425f02, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2023-12-15T01:14:04.327942Z, time_modified: 2023-12-15T01:14:04.327942Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "6a0b318c733ecc8fe690fee9446b5a046a1e6779347663bbc3588532a4ce10cc" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: bcd56901-6236-45d5-99fd-58ae6b425f02, db_silo: Silo { identity: SiloIdentity { id: bcd56901-6236-45d5-99fd-58ae6b425f02, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2023-12-15T01:14:04.327942Z, time_modified: 2023-12-15T01:14:04.327942Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "6a0b318c733ecc8fe690fee9446b5a046a1e6779347663bbc3588532a4ce10cc" }] }) } file = nexus/src/app/background/external_endpoints.rs:105
9302023-12-15T01:20:49.874ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 22.665561ms iteration = 9
9312023-12-15T01:20:49.889ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } dns_group = internal
9322023-12-15T01:20:49.889ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 38.218326ms iteration = 9
9332023-12-15T01:20:49.892ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
9342023-12-15T01:20:49.892ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 1433420e-118d-4f4e-b816-04dd2f29c21e dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
9352023-12-15T01:20:49.892ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
9362023-12-15T01:20:49.892ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
9372023-12-15T01:20:49.893ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9382023-12-15T01:20:49.895ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
9392023-12-15T01:20:49.895ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
9402023-12-15T01:20:49.896ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
9412023-12-15T01:20:49.897ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 1433420e-118d-4f4e-b816-04dd2f29c21e dns_zone_name = control-plane.oxide.internal found_so_far = 51 version = 1
9422023-12-15T01:20:49.897ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
9432023-12-15T01:20:49.897ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2023-12-15 01:14:03.961426 UTC dns_group = internal generation = 1
9442023-12-15T01:20:49.897ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 48.278307ms iteration = 9
9452023-12-15T01:20:49.912ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
9462023-12-15T01:20:49.912ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
9472023-12-15T01:20:49.971ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
9482023-12-15T01:20:49.972ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
9492023-12-15T01:20:49.972ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::8)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "6469c1f7-5610-4bcf-a301-dc37d23f1e93", "date": "Fri, 15 Dec 2023 01:20:49 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
9502023-12-15T01:20:49.972ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/dns_propagation.rs:173 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
9512023-12-15T01:20:49.974ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::9)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "34e12cf1-c96a-4854-a648-f866f8ede49e", "date": "Fri, 15 Dec 2023 01:20:49 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
9522023-12-15T01:20:49.974ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/dns_propagation.rs:173 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
9532023-12-15T01:20:49.975ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/dns_propagation.rs:110 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
9542023-12-15T01:20:49.975ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 123.697553ms iteration = 11
9552023-12-15T01:20:50.001ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
9562023-12-15T01:20:50.002ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:2::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "525344e5-bb18-4221-8c8c-b92f98eac4f2", "date": "Fri, 15 Dec 2023 01:20:49 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
9572023-12-15T01:20:50.002ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/dns_propagation.rs:173 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
9582023-12-15T01:20:50.003ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:1::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "a18f8aa2-6b81-4ac6-bd11-4ef1ea56b50c", "date": "Fri, 15 Dec 2023 01:20:49 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
9592023-12-15T01:20:50.003ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/dns_propagation.rs:173 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
9602023-12-15T01:20:50.004ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:3::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "7de6a8ff-c451-4afd-91ac-1130654b4a45", "date": "Fri, 15 Dec 2023 01:20:49 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
9612023-12-15T01:20:50.004ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/dns_propagation.rs:173 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
9622023-12-15T01:20:50.005ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/dns_propagation.rs:110 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
9632023-12-15T01:20:50.005ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 153.979869ms iteration = 11
9642023-12-15T01:20:54.159ZINFOnexus: received new runtime state from sled agent file = nexus/src/app/instance.rs:1392 instance_id = 4aa645b6-9ac6-43e1-afd5-dec8793ba69d instance_state = InstanceRuntimeState { propolis_id: Some(7bfbb4c7-9544-4c62-b705-404d31141456), dst_propolis_id: None, migration_id: None, gen: Generation(3), time_updated: 2023-12-15T01:20:29.087700138Z } propolis_id = 7bfbb4c7-9544-4c62-b705-404d31141456 vmm_state = VmmRuntimeState { state: Stopping, gen: Generation(5), time_updated: 2023-12-15T01:20:54.158755402Z }
9652023-12-15T01:20:54.159ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:47834 req_id = 0ec4f3b8-8e76-4df7-8658-44624f015bc9 roles = RoleSet { roles: {} } uri = /instances/4aa645b6-9ac6-43e1-afd5-dec8793ba69d
9662023-12-15T01:20:54.159ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:47834 req_id = 0ec4f3b8-8e76-4df7-8658-44624f015bc9 resource = Database result = Ok(()) uri = /instances/4aa645b6-9ac6-43e1-afd5-dec8793ba69d
9672023-12-15T01:20:54.161ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:47834 req_id = 0ec4f3b8-8e76-4df7-8658-44624f015bc9 roles = RoleSet { roles: {} } uri = /instances/4aa645b6-9ac6-43e1-afd5-dec8793ba69d
9682023-12-15T01:20:54.161ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:47834 req_id = 0ec4f3b8-8e76-4df7-8658-44624f015bc9 resource = Database result = Ok(()) uri = /instances/4aa645b6-9ac6-43e1-afd5-dec8793ba69d
9692023-12-15T01:20:54.162ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:47834 req_id = 0ec4f3b8-8e76-4df7-8658-44624f015bc9 roles = RoleSet { roles: {} } uri = /instances/4aa645b6-9ac6-43e1-afd5-dec8793ba69d
9702023-12-15T01:20:54.162ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:47834 req_id = 0ec4f3b8-8e76-4df7-8658-44624f015bc9 resource = Database result = Ok(()) uri = /instances/4aa645b6-9ac6-43e1-afd5-dec8793ba69d
9712023-12-15T01:20:54.163ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:47834 req_id = 0ec4f3b8-8e76-4df7-8658-44624f015bc9 roles = RoleSet { roles: {} } uri = /instances/4aa645b6-9ac6-43e1-afd5-dec8793ba69d
9722023-12-15T01:20:54.164ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:47834 req_id = 0ec4f3b8-8e76-4df7-8658-44624f015bc9 resource = Database result = Ok(()) uri = /instances/4aa645b6-9ac6-43e1-afd5-dec8793ba69d
9732023-12-15T01:20:54.165ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:47834 req_id = 0ec4f3b8-8e76-4df7-8658-44624f015bc9 roles = RoleSet { roles: {} } uri = /instances/4aa645b6-9ac6-43e1-afd5-dec8793ba69d
9742023-12-15T01:20:54.165ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:47834 req_id = 0ec4f3b8-8e76-4df7-8658-44624f015bc9 resource = Database result = Ok(()) uri = /instances/4aa645b6-9ac6-43e1-afd5-dec8793ba69d
9752023-12-15T01:20:54.166ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:47834 req_id = 0ec4f3b8-8e76-4df7-8658-44624f015bc9 roles = RoleSet { roles: {} } uri = /instances/4aa645b6-9ac6-43e1-afd5-dec8793ba69d
9762023-12-15T01:20:54.166ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:47834 req_id = 0ec4f3b8-8e76-4df7-8658-44624f015bc9 resource = Database result = Ok(()) uri = /instances/4aa645b6-9ac6-43e1-afd5-dec8793ba69d
9772023-12-15T01:20:54.167ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:47834 req_id = 0ec4f3b8-8e76-4df7-8658-44624f015bc9 roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } uri = /instances/4aa645b6-9ac6-43e1-afd5-dec8793ba69d
9782023-12-15T01:20:54.170ZDEBGnexus (dropshot_internal): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:47834 req_id = 0ec4f3b8-8e76-4df7-8658-44624f015bc9 resource = Instance { parent: Project { parent: Silo { parent: Fleet, key: bcd56901-6236-45d5-99fd-58ae6b425f02, lookup_type: ById(bcd56901-6236-45d5-99fd-58ae6b425f02) }, key: 5cde8df0-a9e4-46c4-825b-920baa50c1ca, lookup_type: ById(5cde8df0-a9e4-46c4-825b-920baa50c1ca) }, key: 4aa645b6-9ac6-43e1-afd5-dec8793ba69d, lookup_type: ById(4aa645b6-9ac6-43e1-afd5-dec8793ba69d) } result = Ok(()) uri = /instances/4aa645b6-9ac6-43e1-afd5-dec8793ba69d
9792023-12-15T01:20:54.170ZDEBGnexus: instance state generation already advanced, won't touch network config instance_id = 4aa645b6-9ac6-43e1-afd5-dec8793ba69d
9802023-12-15T01:20:54.213ZINFOnexus: instance and vmm updated by sled agent file = nexus/src/app/instance.rs:1528 instance_id = 4aa645b6-9ac6-43e1-afd5-dec8793ba69d instance_updated = false propolis_id = 7bfbb4c7-9544-4c62-b705-404d31141456 vmm_updated = true
9812023-12-15T01:20:54.213ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 54295 local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:47834 req_id = 0ec4f3b8-8e76-4df7-8658-44624f015bc9 response_code = 204 uri = /instances/4aa645b6-9ac6-43e1-afd5-dec8793ba69d
9822023-12-15T01:20:54.213ZINFOnexus: received new runtime state from sled agent file = nexus/src/app/instance.rs:1392 instance_id = 4aa645b6-9ac6-43e1-afd5-dec8793ba69d instance_state = InstanceRuntimeState { propolis_id: Some(7bfbb4c7-9544-4c62-b705-404d31141456), dst_propolis_id: None, migration_id: None, gen: Generation(3), time_updated: 2023-12-15T01:20:29.087700138Z } propolis_id = 7bfbb4c7-9544-4c62-b705-404d31141456 vmm_state = VmmRuntimeState { state: Stopping, gen: Generation(6), time_updated: 2023-12-15T01:20:54.213537297Z }
9832023-12-15T01:20:54.213ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:47834 req_id = 8337dc10-e4a7-4ad5-8e05-a6486adee899 roles = RoleSet { roles: {} } uri = /instances/4aa645b6-9ac6-43e1-afd5-dec8793ba69d
9842023-12-15T01:20:54.213ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:47834 req_id = 8337dc10-e4a7-4ad5-8e05-a6486adee899 resource = Database result = Ok(()) uri = /instances/4aa645b6-9ac6-43e1-afd5-dec8793ba69d
9852023-12-15T01:20:54.215ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:47834 req_id = 8337dc10-e4a7-4ad5-8e05-a6486adee899 roles = RoleSet { roles: {} } uri = /instances/4aa645b6-9ac6-43e1-afd5-dec8793ba69d
9862023-12-15T01:20:54.215ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:47834 req_id = 8337dc10-e4a7-4ad5-8e05-a6486adee899 resource = Database result = Ok(()) uri = /instances/4aa645b6-9ac6-43e1-afd5-dec8793ba69d
9872023-12-15T01:20:54.217ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:47834 req_id = 8337dc10-e4a7-4ad5-8e05-a6486adee899 roles = RoleSet { roles: {} } uri = /instances/4aa645b6-9ac6-43e1-afd5-dec8793ba69d
9882023-12-15T01:20:54.217ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:47834 req_id = 8337dc10-e4a7-4ad5-8e05-a6486adee899 resource = Database result = Ok(()) uri = /instances/4aa645b6-9ac6-43e1-afd5-dec8793ba69d
9892023-12-15T01:20:54.218ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:47834 req_id = 8337dc10-e4a7-4ad5-8e05-a6486adee899 roles = RoleSet { roles: {} } uri = /instances/4aa645b6-9ac6-43e1-afd5-dec8793ba69d
9902023-12-15T01:20:54.218ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:47834 req_id = 8337dc10-e4a7-4ad5-8e05-a6486adee899 resource = Database result = Ok(()) uri = /instances/4aa645b6-9ac6-43e1-afd5-dec8793ba69d
9912023-12-15T01:20:54.219ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:47834 req_id = 8337dc10-e4a7-4ad5-8e05-a6486adee899 roles = RoleSet { roles: {} } uri = /instances/4aa645b6-9ac6-43e1-afd5-dec8793ba69d
9922023-12-15T01:20:54.219ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:47834 req_id = 8337dc10-e4a7-4ad5-8e05-a6486adee899 resource = Database result = Ok(()) uri = /instances/4aa645b6-9ac6-43e1-afd5-dec8793ba69d
9932023-12-15T01:20:54.220ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:47834 req_id = 8337dc10-e4a7-4ad5-8e05-a6486adee899 roles = RoleSet { roles: {} } uri = /instances/4aa645b6-9ac6-43e1-afd5-dec8793ba69d
9942023-12-15T01:20:54.221ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:47834 req_id = 8337dc10-e4a7-4ad5-8e05-a6486adee899 resource = Database result = Ok(()) uri = /instances/4aa645b6-9ac6-43e1-afd5-dec8793ba69d
9952023-12-15T01:20:54.222ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:47834 req_id = 8337dc10-e4a7-4ad5-8e05-a6486adee899 roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } uri = /instances/4aa645b6-9ac6-43e1-afd5-dec8793ba69d
9962023-12-15T01:20:54.224ZDEBGnexus (dropshot_internal): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:47834 req_id = 8337dc10-e4a7-4ad5-8e05-a6486adee899 resource = Instance { parent: Project { parent: Silo { parent: Fleet, key: bcd56901-6236-45d5-99fd-58ae6b425f02, lookup_type: ById(bcd56901-6236-45d5-99fd-58ae6b425f02) }, key: 5cde8df0-a9e4-46c4-825b-920baa50c1ca, lookup_type: ById(5cde8df0-a9e4-46c4-825b-920baa50c1ca) }, key: 4aa645b6-9ac6-43e1-afd5-dec8793ba69d, lookup_type: ById(4aa645b6-9ac6-43e1-afd5-dec8793ba69d) } result = Ok(()) uri = /instances/4aa645b6-9ac6-43e1-afd5-dec8793ba69d
9972023-12-15T01:20:54.224ZDEBGnexus: instance state generation already advanced, won't touch network config instance_id = 4aa645b6-9ac6-43e1-afd5-dec8793ba69d
9982023-12-15T01:20:54.232ZINFOnexus: instance and vmm updated by sled agent file = nexus/src/app/instance.rs:1528 instance_id = 4aa645b6-9ac6-43e1-afd5-dec8793ba69d instance_updated = false propolis_id = 7bfbb4c7-9544-4c62-b705-404d31141456 vmm_updated = true
9992023-12-15T01:20:54.232ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 18953 local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:47834 req_id = 8337dc10-e4a7-4ad5-8e05-a6486adee899 response_code = 204 uri = /instances/4aa645b6-9ac6-43e1-afd5-dec8793ba69d
10002023-12-15T01:20:54.899ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:46101 req_id = cd709683-2669-4619-b49a-a1696ffbb46b roles = RoleSet { roles: {} } uri = /nat/ipv4/changeset/2?limit=100
10012023-12-15T01:20:54.899ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:46101 req_id = cd709683-2669-4619-b49a-a1696ffbb46b resource = Database result = Ok(()) uri = /nat/ipv4/changeset/2?limit=100
10022023-12-15T01:20:54.902ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 3437 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:46101 req_id = cd709683-2669-4619-b49a-a1696ffbb46b response_code = 200 uri = /nat/ipv4/changeset/2?limit=100
10032023-12-15T01:20:54.903ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:46101 req_id = 7bd89b88-6fac-4372-838d-c0661896cea0 roles = RoleSet { roles: {} } uri = /nat/ipv4/changeset/4?limit=100
10042023-12-15T01:20:54.904ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:46101 req_id = 7bd89b88-6fac-4372-838d-c0661896cea0 resource = Database result = Ok(()) uri = /nat/ipv4/changeset/4?limit=100
10052023-12-15T01:20:54.905ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 2629 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:46101 req_id = 7bd89b88-6fac-4372-838d-c0661896cea0 response_code = 200 uri = /nat/ipv4/changeset/4?limit=100
10062023-12-15T01:20:55.906ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:46101 req_id = dea4a7ea-9526-4fd3-9365-113162ac4f06 roles = RoleSet { roles: {} } uri = /nat/ipv4/changeset/4?limit=100
10072023-12-15T01:20:55.906ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:46101 req_id = dea4a7ea-9526-4fd3-9365-113162ac4f06 resource = Database result = Ok(()) uri = /nat/ipv4/changeset/4?limit=100
10082023-12-15T01:20:55.908ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 2215 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:46101 req_id = dea4a7ea-9526-4fd3-9365-113162ac4f06 response_code = 200 uri = /nat/ipv4/changeset/4?limit=100
10092023-12-15T01:20:56.143ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 716 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:60064 req_id = 87faa55c-01da-4283-bb13-1a21cb5d05ed response_code = 200 uri = /metrics/collect/700e7b49-f840-40d5-92e8-83ab758ea8a7
10102023-12-15T01:20:56.563ZINFOnexus: received new runtime state from sled agent file = nexus/src/app/instance.rs:1392 instance_id = 4aa645b6-9ac6-43e1-afd5-dec8793ba69d instance_state = InstanceRuntimeState { propolis_id: None, dst_propolis_id: None, migration_id: None, gen: Generation(5), time_updated: 2023-12-15T01:20:54.351732428Z } propolis_id = 7bfbb4c7-9544-4c62-b705-404d31141456 vmm_state = VmmRuntimeState { state: Destroyed, gen: Generation(7), time_updated: 2023-12-15T01:20:54.351732428Z }
10112023-12-15T01:20:56.563ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:47834 req_id = 851a543b-8c0b-454d-9408-251bf234cee1 roles = RoleSet { roles: {} } uri = /instances/4aa645b6-9ac6-43e1-afd5-dec8793ba69d
10122023-12-15T01:20:56.563ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:47834 req_id = 851a543b-8c0b-454d-9408-251bf234cee1 resource = Database result = Ok(()) uri = /instances/4aa645b6-9ac6-43e1-afd5-dec8793ba69d
10132023-12-15T01:20:56.565ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:47834 req_id = 851a543b-8c0b-454d-9408-251bf234cee1 roles = RoleSet { roles: {} } uri = /instances/4aa645b6-9ac6-43e1-afd5-dec8793ba69d
10142023-12-15T01:20:56.565ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:47834 req_id = 851a543b-8c0b-454d-9408-251bf234cee1 resource = Database result = Ok(()) uri = /instances/4aa645b6-9ac6-43e1-afd5-dec8793ba69d
10152023-12-15T01:20:56.566ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:47834 req_id = 851a543b-8c0b-454d-9408-251bf234cee1 roles = RoleSet { roles: {} } uri = /instances/4aa645b6-9ac6-43e1-afd5-dec8793ba69d
10162023-12-15T01:20:56.566ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:47834 req_id = 851a543b-8c0b-454d-9408-251bf234cee1 resource = Database result = Ok(()) uri = /instances/4aa645b6-9ac6-43e1-afd5-dec8793ba69d
10172023-12-15T01:20:56.568ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:47834 req_id = 851a543b-8c0b-454d-9408-251bf234cee1 roles = RoleSet { roles: {} } uri = /instances/4aa645b6-9ac6-43e1-afd5-dec8793ba69d
10182023-12-15T01:20:56.568ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:47834 req_id = 851a543b-8c0b-454d-9408-251bf234cee1 resource = Database result = Ok(()) uri = /instances/4aa645b6-9ac6-43e1-afd5-dec8793ba69d
10192023-12-15T01:20:56.570ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:47834 req_id = 851a543b-8c0b-454d-9408-251bf234cee1 roles = RoleSet { roles: {} } uri = /instances/4aa645b6-9ac6-43e1-afd5-dec8793ba69d
10202023-12-15T01:20:56.570ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:47834 req_id = 851a543b-8c0b-454d-9408-251bf234cee1 resource = Database result = Ok(()) uri = /instances/4aa645b6-9ac6-43e1-afd5-dec8793ba69d
10212023-12-15T01:20:56.571ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:47834 req_id = 851a543b-8c0b-454d-9408-251bf234cee1 roles = RoleSet { roles: {} } uri = /instances/4aa645b6-9ac6-43e1-afd5-dec8793ba69d
10222023-12-15T01:20:56.571ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:47834 req_id = 851a543b-8c0b-454d-9408-251bf234cee1 resource = Database result = Ok(()) uri = /instances/4aa645b6-9ac6-43e1-afd5-dec8793ba69d
10232023-12-15T01:20:56.572ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:47834 req_id = 851a543b-8c0b-454d-9408-251bf234cee1 roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } uri = /instances/4aa645b6-9ac6-43e1-afd5-dec8793ba69d
10242023-12-15T01:20:56.575ZDEBGnexus (dropshot_internal): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:47834 req_id = 851a543b-8c0b-454d-9408-251bf234cee1 resource = Instance { parent: Project { parent: Silo { parent: Fleet, key: bcd56901-6236-45d5-99fd-58ae6b425f02, lookup_type: ById(bcd56901-6236-45d5-99fd-58ae6b425f02) }, key: 5cde8df0-a9e4-46c4-825b-920baa50c1ca, lookup_type: ById(5cde8df0-a9e4-46c4-825b-920baa50c1ca) }, key: 4aa645b6-9ac6-43e1-afd5-dec8793ba69d, lookup_type: ById(4aa645b6-9ac6-43e1-afd5-dec8793ba69d) } result = Ok(()) uri = /instances/4aa645b6-9ac6-43e1-afd5-dec8793ba69d
10252023-12-15T01:20:56.575ZINFOnexus: instance cleared its Propolis ID, cleaning network config file = nexus/src/app/instance_network.rs:626 instance_id = 4aa645b6-9ac6-43e1-afd5-dec8793ba69d propolis_id = Some(7bfbb4c7-9544-4c62-b705-404d31141456)
10262023-12-15T01:20:56.575ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:47834 req_id = 851a543b-8c0b-454d-9408-251bf234cee1 roles = RoleSet { roles: {} } uri = /instances/4aa645b6-9ac6-43e1-afd5-dec8793ba69d
10272023-12-15T01:20:56.575ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:47834 req_id = 851a543b-8c0b-454d-9408-251bf234cee1 resource = Database result = Ok(()) uri = /instances/4aa645b6-9ac6-43e1-afd5-dec8793ba69d
10282023-12-15T01:20:56.577ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:47834 req_id = 851a543b-8c0b-454d-9408-251bf234cee1 roles = RoleSet { roles: {} } uri = /instances/4aa645b6-9ac6-43e1-afd5-dec8793ba69d
10292023-12-15T01:20:56.577ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:47834 req_id = 851a543b-8c0b-454d-9408-251bf234cee1 resource = Database result = Ok(()) uri = /instances/4aa645b6-9ac6-43e1-afd5-dec8793ba69d
10302023-12-15T01:20:56.579ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:47834 req_id = 851a543b-8c0b-454d-9408-251bf234cee1 roles = RoleSet { roles: {} } uri = /instances/4aa645b6-9ac6-43e1-afd5-dec8793ba69d
10312023-12-15T01:20:56.579ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:47834 req_id = 851a543b-8c0b-454d-9408-251bf234cee1 resource = Database result = Ok(()) uri = /instances/4aa645b6-9ac6-43e1-afd5-dec8793ba69d
10322023-12-15T01:20:56.580ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:47834 req_id = 851a543b-8c0b-454d-9408-251bf234cee1 roles = RoleSet { roles: {} } uri = /instances/4aa645b6-9ac6-43e1-afd5-dec8793ba69d
10332023-12-15T01:20:56.580ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:47834 req_id = 851a543b-8c0b-454d-9408-251bf234cee1 resource = Database result = Ok(()) uri = /instances/4aa645b6-9ac6-43e1-afd5-dec8793ba69d
10342023-12-15T01:20:56.581ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:47834 req_id = 851a543b-8c0b-454d-9408-251bf234cee1 roles = RoleSet { roles: {} } uri = /instances/4aa645b6-9ac6-43e1-afd5-dec8793ba69d
10352023-12-15T01:20:56.581ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:47834 req_id = 851a543b-8c0b-454d-9408-251bf234cee1 resource = Database result = Ok(()) uri = /instances/4aa645b6-9ac6-43e1-afd5-dec8793ba69d
10362023-12-15T01:20:56.582ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:47834 req_id = 851a543b-8c0b-454d-9408-251bf234cee1 roles = RoleSet { roles: {} } uri = /instances/4aa645b6-9ac6-43e1-afd5-dec8793ba69d
10372023-12-15T01:20:56.583ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:47834 req_id = 851a543b-8c0b-454d-9408-251bf234cee1 resource = Database result = Ok(()) uri = /instances/4aa645b6-9ac6-43e1-afd5-dec8793ba69d
10382023-12-15T01:20:56.584ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:47834 req_id = 851a543b-8c0b-454d-9408-251bf234cee1 roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } uri = /instances/4aa645b6-9ac6-43e1-afd5-dec8793ba69d
10392023-12-15T01:20:56.586ZDEBGnexus (dropshot_internal): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:47834 req_id = 851a543b-8c0b-454d-9408-251bf234cee1 resource = Instance { parent: Project { parent: Silo { parent: Fleet, key: bcd56901-6236-45d5-99fd-58ae6b425f02, lookup_type: ById(bcd56901-6236-45d5-99fd-58ae6b425f02) }, key: 5cde8df0-a9e4-46c4-825b-920baa50c1ca, lookup_type: ById(5cde8df0-a9e4-46c4-825b-920baa50c1ca) }, key: 4aa645b6-9ac6-43e1-afd5-dec8793ba69d, lookup_type: ById(4aa645b6-9ac6-43e1-afd5-dec8793ba69d) } result = Ok(()) uri = /instances/4aa645b6-9ac6-43e1-afd5-dec8793ba69d
10402023-12-15T01:20:56.586ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:47834 req_id = 851a543b-8c0b-454d-9408-251bf234cee1 roles = RoleSet { roles: {} } uri = /instances/4aa645b6-9ac6-43e1-afd5-dec8793ba69d
10412023-12-15T01:20:56.587ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:47834 req_id = 851a543b-8c0b-454d-9408-251bf234cee1 resource = Database result = Ok(()) uri = /instances/4aa645b6-9ac6-43e1-afd5-dec8793ba69d
10422023-12-15T01:20:56.588ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:47834 req_id = 851a543b-8c0b-454d-9408-251bf234cee1 roles = RoleSet { roles: {} } uri = /instances/4aa645b6-9ac6-43e1-afd5-dec8793ba69d
10432023-12-15T01:20:56.588ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:47834 req_id = 851a543b-8c0b-454d-9408-251bf234cee1 resource = Database result = Ok(()) uri = /instances/4aa645b6-9ac6-43e1-afd5-dec8793ba69d
10442023-12-15T01:20:56.589ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:47834 req_id = 851a543b-8c0b-454d-9408-251bf234cee1 roles = RoleSet { roles: {} } uri = /instances/4aa645b6-9ac6-43e1-afd5-dec8793ba69d
10452023-12-15T01:20:56.589ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:47834 req_id = 851a543b-8c0b-454d-9408-251bf234cee1 resource = Database result = Ok(()) uri = /instances/4aa645b6-9ac6-43e1-afd5-dec8793ba69d
10462023-12-15T01:20:56.590ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:47834 req_id = 851a543b-8c0b-454d-9408-251bf234cee1 roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } uri = /instances/4aa645b6-9ac6-43e1-afd5-dec8793ba69d
10472023-12-15T01:20:56.593ZDEBGnexus (dropshot_internal): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:47834 req_id = 851a543b-8c0b-454d-9408-251bf234cee1 resource = Instance { parent: Project { parent: Silo { parent: Fleet, key: bcd56901-6236-45d5-99fd-58ae6b425f02, lookup_type: ById(bcd56901-6236-45d5-99fd-58ae6b425f02) }, key: 5cde8df0-a9e4-46c4-825b-920baa50c1ca, lookup_type: ById(5cde8df0-a9e4-46c4-825b-920baa50c1ca) }, key: 4aa645b6-9ac6-43e1-afd5-dec8793ba69d, lookup_type: ById(4aa645b6-9ac6-43e1-afd5-dec8793ba69d) } result = Ok(()) uri = /instances/4aa645b6-9ac6-43e1-afd5-dec8793ba69d
10482023-12-15T01:20:56.593ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:47834 req_id = 851a543b-8c0b-454d-9408-251bf234cee1 roles = RoleSet { roles: {} } uri = /instances/4aa645b6-9ac6-43e1-afd5-dec8793ba69d
10492023-12-15T01:20:56.593ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:47834 req_id = 851a543b-8c0b-454d-9408-251bf234cee1 resource = Database result = Ok(()) uri = /instances/4aa645b6-9ac6-43e1-afd5-dec8793ba69d
10502023-12-15T01:20:56.597ZDEBGnexus (InstanceAllocator): roles roles = RoleSet { roles: {} }
10512023-12-15T01:20:56.597ZDEBGnexus (InstanceAllocator): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Database result = Ok(())
10522023-12-15T01:20:56.598ZDEBGnexus (InstanceAllocator): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "viewer")} }
10532023-12-15T01:20:56.599ZDEBGnexus (InstanceAllocator): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Fleet result = Ok(())
10542023-12-15T01:20:56.599ZDEBGnexus (InstanceAllocator): roles roles = RoleSet { roles: {} }
10552023-12-15T01:20:56.599ZDEBGnexus (InstanceAllocator): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Database result = Ok(())
10562023-12-15T01:20:56.601ZDEBGnexus (InstanceAllocator): roles roles = RoleSet { roles: {} }
10572023-12-15T01:20:56.601ZDEBGnexus (InstanceAllocator): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Database result = Ok(())
10582023-12-15T01:20:56.603ZDEBGnexus (InstanceAllocator): roles roles = RoleSet { roles: {} }
10592023-12-15T01:20:56.603ZDEBGnexus (InstanceAllocator): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Database result = Ok(())
10602023-12-15T01:20:56.604ZDEBGnexus (InstanceAllocator): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "viewer")} }
10612023-12-15T01:20:56.606ZDEBGnexus (InstanceAllocator): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Sled { parent: Fleet, key: 78829061-d92a-47ac-bcd6-186a0b9f0582, lookup_type: ById(78829061-d92a-47ac-bcd6-186a0b9f0582) } result = Ok(())
10622023-12-15T01:20:56.636ZDEBGnexus: client request SledAgent = 78829061-d92a-47ac-bcd6-186a0b9f0582 body = Some(Body) method = DELETE uri = http://[fd00:1122:3344:101::1]:12345/v2p/985afe56-fc39-4541-abcc-7bbac13c05ae
10632023-12-15T01:20:56.636ZDEBGnexus: client response SledAgent = 78829061-d92a-47ac-bcd6-186a0b9f0582 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/v2p/985afe56-fc39-4541-abcc-7bbac13c05ae", query: None, fragment: None }, status: 204, headers: {"x-request-id": "ea64650c-04b0-4a01-b68a-1be990224ca2", "date": "Fri, 15 Dec 2023 01:20:56 GMT"} })
10642023-12-15T01:20:56.638ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:47834 req_id = 851a543b-8c0b-454d-9408-251bf234cee1 roles = RoleSet { roles: {} } uri = /instances/4aa645b6-9ac6-43e1-afd5-dec8793ba69d
10652023-12-15T01:20:56.638ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:47834 req_id = 851a543b-8c0b-454d-9408-251bf234cee1 resource = Database result = Ok(()) uri = /instances/4aa645b6-9ac6-43e1-afd5-dec8793ba69d
10662023-12-15T01:20:56.643ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:47834 req_id = 851a543b-8c0b-454d-9408-251bf234cee1 roles = RoleSet { roles: {} } uri = /instances/4aa645b6-9ac6-43e1-afd5-dec8793ba69d
10672023-12-15T01:20:56.644ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:47834 req_id = 851a543b-8c0b-454d-9408-251bf234cee1 resource = Database result = Ok(()) uri = /instances/4aa645b6-9ac6-43e1-afd5-dec8793ba69d
10682023-12-15T01:20:56.645ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:47834 req_id = 851a543b-8c0b-454d-9408-251bf234cee1 roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } uri = /instances/4aa645b6-9ac6-43e1-afd5-dec8793ba69d
10692023-12-15T01:20:56.647ZDEBGnexus (dropshot_internal): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:47834 req_id = 851a543b-8c0b-454d-9408-251bf234cee1 resource = Fleet result = Ok(()) uri = /instances/4aa645b6-9ac6-43e1-afd5-dec8793ba69d
10702023-12-15T01:20:56.647ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:47834 req_id = 851a543b-8c0b-454d-9408-251bf234cee1 roles = RoleSet { roles: {} } uri = /instances/4aa645b6-9ac6-43e1-afd5-dec8793ba69d
10712023-12-15T01:20:56.647ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:47834 req_id = 851a543b-8c0b-454d-9408-251bf234cee1 resource = Database result = Ok(()) uri = /instances/4aa645b6-9ac6-43e1-afd5-dec8793ba69d
10722023-12-15T01:20:56.651ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:47834 req_id = 851a543b-8c0b-454d-9408-251bf234cee1 roles = RoleSet { roles: {} } uri = /instances/4aa645b6-9ac6-43e1-afd5-dec8793ba69d
10732023-12-15T01:20:56.651ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:47834 req_id = 851a543b-8c0b-454d-9408-251bf234cee1 resource = Database result = Ok(()) uri = /instances/4aa645b6-9ac6-43e1-afd5-dec8793ba69d
10742023-12-15T01:20:56.658ZWARNnexus: no matching nat entries to soft delete file = nexus/src/app/instance_network.rs:543
10752023-12-15T01:20:56.658ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:47834 req_id = 851a543b-8c0b-454d-9408-251bf234cee1 roles = RoleSet { roles: {} } uri = /instances/4aa645b6-9ac6-43e1-afd5-dec8793ba69d
10762023-12-15T01:20:56.658ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:47834 req_id = 851a543b-8c0b-454d-9408-251bf234cee1 resource = Database result = Ok(()) uri = /instances/4aa645b6-9ac6-43e1-afd5-dec8793ba69d
10772023-12-15T01:20:56.659ZWARNnexus: no matching nat entries to soft delete file = nexus/src/app/instance_network.rs:543
10782023-12-15T01:20:56.659ZDEBGnexus: notifying dendrite of updates instance_id = 4aa645b6-9ac6-43e1-afd5-dec8793ba69d switch = switch0
10792023-12-15T01:20:56.659ZDEBGnexus (DpdClient): client request body = None method = POST uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/trigger
10802023-12-15T01:20:56.660ZDEBGnexus (DpdClient): client response result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/trigger", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "bc6f4520-c690-4e7e-9dc2-06b26bad1dcc", "content-length": "4", "date": "Fri, 15 Dec 2023 01:20:55 GMT"} })
10812023-12-15T01:20:56.660ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:47834 req_id = 851a543b-8c0b-454d-9408-251bf234cee1 roles = RoleSet { roles: {} } uri = /instances/4aa645b6-9ac6-43e1-afd5-dec8793ba69d
10822023-12-15T01:20:56.660ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:47834 req_id = 851a543b-8c0b-454d-9408-251bf234cee1 resource = Database result = Ok(()) uri = /instances/4aa645b6-9ac6-43e1-afd5-dec8793ba69d
10832023-12-15T01:20:56.675ZDEBGnexus: deleted metric producer assignment collector_id = 127efcba-5993-4577-bcb2-f4a98a4520bb producer_id = 4aa645b6-9ac6-43e1-afd5-dec8793ba69d
10842023-12-15T01:20:56.708ZINFOnexus: registered oximeter collector client file = nexus/src/app/oximeter.rs:374 id = 4aa645b6-9ac6-43e1-afd5-dec8793ba69d
10852023-12-15T01:20:56.708ZDEBGnexus: client request body = None method = DELETE oximeter-collector = 4aa645b6-9ac6-43e1-afd5-dec8793ba69d uri = http://[fd00:1122:3344:101::d]:12223/producers/4aa645b6-9ac6-43e1-afd5-dec8793ba69d
10862023-12-15T01:20:56.714ZDEBGnexus: client response oximeter-collector = 4aa645b6-9ac6-43e1-afd5-dec8793ba69d result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::d)), port: Some(12223), path: "/producers/4aa645b6-9ac6-43e1-afd5-dec8793ba69d", query: None, fragment: None }, status: 204, headers: {"x-request-id": "c6a80d54-1c34-44cc-b723-65d78dff4f51", "date": "Fri, 15 Dec 2023 01:20:56 GMT"} })
10872023-12-15T01:20:56.714ZDEBGnexus: successfully deleted producer from collector address = [fd00:1122:3344:101::d]:12223 collector_id = 127efcba-5993-4577-bcb2-f4a98a4520bb producer_id = 4aa645b6-9ac6-43e1-afd5-dec8793ba69d
10882023-12-15T01:20:56.804ZINFOnexus: vmm is terminated, cleaning up resources file = nexus/src/app/instance.rs:1505 instance_id = 4aa645b6-9ac6-43e1-afd5-dec8793ba69d propolis_id = 7bfbb4c7-9544-4c62-b705-404d31141456
10892023-12-15T01:20:56.804ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:47834 req_id = 851a543b-8c0b-454d-9408-251bf234cee1 roles = RoleSet { roles: {} } uri = /instances/4aa645b6-9ac6-43e1-afd5-dec8793ba69d
10902023-12-15T01:20:56.804ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:47834 req_id = 851a543b-8c0b-454d-9408-251bf234cee1 resource = Database result = Ok(()) uri = /instances/4aa645b6-9ac6-43e1-afd5-dec8793ba69d
10912023-12-15T01:20:56.809ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:47834 req_id = 851a543b-8c0b-454d-9408-251bf234cee1 roles = RoleSet { roles: {} } uri = /instances/4aa645b6-9ac6-43e1-afd5-dec8793ba69d
10922023-12-15T01:20:56.809ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:47834 req_id = 851a543b-8c0b-454d-9408-251bf234cee1 resource = Database result = Ok(()) uri = /instances/4aa645b6-9ac6-43e1-afd5-dec8793ba69d
10932023-12-15T01:20:56.853ZINFOnexus: instance and vmm updated by sled agent file = nexus/src/app/instance.rs:1528 instance_id = 4aa645b6-9ac6-43e1-afd5-dec8793ba69d instance_updated = true propolis_id = 7bfbb4c7-9544-4c62-b705-404d31141456 vmm_updated = true
10942023-12-15T01:20:56.853ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 290230 local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:47834 req_id = 851a543b-8c0b-454d-9408-251bf234cee1 response_code = 204 uri = /instances/4aa645b6-9ac6-43e1-afd5-dec8793ba69d
10952023-12-15T01:20:56.909ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:46101 req_id = 8c1b3bbb-e6a7-4a90-bb7f-80e466a3ba2d roles = RoleSet { roles: {} } uri = /nat/ipv4/changeset/4?limit=100
10962023-12-15T01:20:56.910ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:46101 req_id = 8c1b3bbb-e6a7-4a90-bb7f-80e466a3ba2d resource = Database result = Ok(()) uri = /nat/ipv4/changeset/4?limit=100
10972023-12-15T01:20:56.911ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 2230 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:46101 req_id = 8c1b3bbb-e6a7-4a90-bb7f-80e466a3ba2d response_code = 200 uri = /nat/ipv4/changeset/4?limit=100
10982023-12-15T01:20:58.913ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:46101 req_id = e6b95fcd-4cd4-42b6-b396-7b8d14d05bd9 roles = RoleSet { roles: {} } uri = /nat/ipv4/changeset/4?limit=100
10992023-12-15T01:20:58.914ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:46101 req_id = e6b95fcd-4cd4-42b6-b396-7b8d14d05bd9 resource = Database result = Ok(()) uri = /nat/ipv4/changeset/4?limit=100
11002023-12-15T01:20:58.915ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 2120 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:46101 req_id = e6b95fcd-4cd4-42b6-b396-7b8d14d05bd9 response_code = 200 uri = /nat/ipv4/changeset/4?limit=100
11012023-12-15T01:21:06.144ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 955 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:60064 req_id = 0ab7026a-b2f3-478a-9cc3-6bf1857aff25 response_code = 200 uri = /metrics/collect/700e7b49-f840-40d5-92e8-83ab758ea8a7
11022023-12-15T01:21:16.143ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 677 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:60064 req_id = 9bddec0a-bc49-4d56-a855-041253cc3f19 response_code = 200 uri = /metrics/collect/700e7b49-f840-40d5-92e8-83ab758ea8a7
11032023-12-15T01:21:19.853ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 16 reason = Timeout
11042023-12-15T01:21:19.853ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
11052023-12-15T01:21:19.853ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 16 reason = Timeout
11062023-12-15T01:21:19.853ZWARNnexus (BackgroundTasks): phantom disk task started background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:50
11072023-12-15T01:21:19.854ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
11082023-12-15T01:21:19.855ZDEBGnexus (DpdClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
11092023-12-15T01:21:19.855ZDEBGnexus (DpdClient): client response result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "30f8145d-9b39-449b-b76a-d609465fc71f", "content-length": "1", "date": "Fri, 15 Dec 2023 01:21:19 GMT"} })
11102023-12-15T01:21:19.855ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
11112023-12-15T01:21:19.855ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
11122023-12-15T01:21:19.856ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 3.199688ms iteration = 16
11132023-12-15T01:21:19.857ZWARNnexus (BackgroundTasks): phantom disk task done background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:96
11142023-12-15T01:21:19.857ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.441121ms iteration = 16
11152023-12-15T01:21:26.143ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 680 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:60064 req_id = e0eaf901-8557-4532-b6a2-a024dddb1f5f response_code = 200 uri = /metrics/collect/700e7b49-f840-40d5-92e8-83ab758ea8a7
11162023-12-15T01:21:27.946ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:46101 req_id = c89dcc3f-a087-406e-948d-7da2193da1b6 roles = RoleSet { roles: {} } uri = /nat/ipv4/changeset/4?limit=100
11172023-12-15T01:21:27.946ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:46101 req_id = c89dcc3f-a087-406e-948d-7da2193da1b6 resource = Database result = Ok(()) uri = /nat/ipv4/changeset/4?limit=100
11182023-12-15T01:21:27.948ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 2711 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:46101 req_id = c89dcc3f-a087-406e-948d-7da2193da1b6 response_code = 200 uri = /nat/ipv4/changeset/4?limit=100
11192023-12-15T01:21:36.144ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 1266 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:60064 req_id = feed4698-4124-4b60-834a-411fed681a7e response_code = 200 uri = /metrics/collect/700e7b49-f840-40d5-92e8-83ab758ea8a7