Max level shown:
1 [ Nov 20 07:50:10 Enabled. ]
2 [ Nov 20 07:50:10 Executing start method ("ctrun -l child -o noorphan,regent /opt/oxide/sled-agent/sled-agent run /opt/oxide/sled-agent/pkg/config.toml &"). ]
3 [ Nov 20 07:50:10 Method "start" exited with status 0. ]
4 [ Nov 20 07:50:10 Rereading configuration. ]
5 [ Nov 20 07:50:10 No 'refresh' method defined. Treating as :true. ]
6 note: configured to log to "/dev/stdout"
72023-11-20T07:50:10.988ZINFOSledAgent (BootstrapAgentStartup): finding links ["net0", "net1"] file = sled-agent/src/bootstrap/pre_server.rs:240
82023-11-20T07:50:11.042ZINFOSledAgent (BootstrapAgentStartup): Starting mg-ddm service addr-objs = [AddrObject { interface: "net0", name: "ll" }, AddrObject { interface: "net1", name: "ll" }] file = sled-agent/src/bootstrap/pre_server.rs:247
92023-11-20T07:50:11.042ZINFOSledAgent (BootstrapAgentStartup): Importing mg-ddm service file = sled-agent/src/bootstrap/maghemite.rs:42 path = /opt/oxide/mg-ddm/pkg/ddm/manifest.xml
102023-11-20T07:50:11.153ZINFOSledAgent (BootstrapAgentStartup): Setting mg-ddm interfaces file = sled-agent/src/bootstrap/maghemite.rs:50 interfaces = ("net0/ll" "net1/ll")
112023-11-20T07:50:11.162ZINFOSledAgent (BootstrapAgentStartup): Enabling mg-ddm service file = sled-agent/src/bootstrap/maghemite.rs:56
122023-11-20T07:50:11.171ZINFOSledAgent (BootstrapAgentStartup): Ensuring zfs key directory exists file = sled-agent/src/bootstrap/pre_server.rs:257 path = /var/run/oxide/
132023-11-20T07:50:11.775ZINFOSledAgent (BootstrapAgentStartup): Sending prefix to ddmd for advertisement DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:91 prefix = Ipv6Prefix { addr: fdb0:18c0:4d0d:a02a::, len: 64 }
142023-11-20T07:50:11.913ZINFOSledAgent: Starting KeyManager file = sled-agent/src/long_running_tasks.rs:117
152023-11-20T07:50:11.913ZINFOSledAgent: Starting StorageManager file = sled-agent/src/long_running_tasks.rs:129
162023-11-20T07:50:11.913ZINFOSledAgent: Starting StorageMonitor file = sled-agent/src/long_running_tasks.rs:141
172023-11-20T07:50:11.914ZINFOSledAgent: Starting HardwareManager file = sled-agent/src/long_running_tasks.rs:162 sled_mode = Scrimlet { asic: SoftNpuZone }
182023-11-20T07:50:11.914ZINFOSledAgent (DumpSetup-archival): DumpSetup poll loop started. file = sled-agent/src/dump_setup.rs:171
192023-11-20T07:50:11.914ZINFOSledAgent (HardwareManager): Creating HardwareManager file = sled-hardware/src/illumos/mod.rs:606
202023-11-20T07:50:11.914ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:532
212023-11-20T07:50:11.917ZINFOSledAgent (HardwareManager): Generated i86pc baseboard Pc { identifier: "57495fb0328c443a91abff25dfea8602", model: "i86pc" } file = sled-hardware/src/illumos/mod.rs:532
222023-11-20T07:50:11.917ZWARNSledAgent (HardwareManager): Device is not a Gimlet (i86pc), proceeding with null hardware view file = sled-hardware/src/illumos/mod.rs:655
232023-11-20T07:50:11.917ZINFOSledAgent: Starting HardwareMonitor file = sled-agent/src/long_running_tasks.rs:176
242023-11-20T07:50:11.917ZINFOSledAgent (HardwareManager): Monitoring for hardware updates file = sled-hardware/src/illumos/mod.rs:713
252023-11-20T07:50:11.917ZINFOSledAgent: Upserting synthetic zpool to Storage Manager: oxi_a462a7f7-b628-40fe-80ff-4e4189e2d62b file = sled-agent/src/long_running_tasks.rs:232
262023-11-20T07:50:11.917ZINFOSledAgent: Upserting synthetic zpool to Storage Manager: oxi_b462a7f7-b628-40fe-80ff-4e4189e2d62b file = sled-agent/src/long_running_tasks.rs:232
272023-11-20T07:50:11.917ZINFOSledAgent: Upserting synthetic zpool to Storage Manager: oxp_d462a7f7-b628-40fe-80ff-4e4189e2d62b file = sled-agent/src/long_running_tasks.rs:232
282023-11-20T07:50:11.917ZINFOSledAgent: Upserting synthetic zpool to Storage Manager: oxp_e4b4dc87-ab46-49fb-a4b4-d361ae214c03 file = sled-agent/src/long_running_tasks.rs:232
292023-11-20T07:50:11.917ZINFOSledAgent: Upserting synthetic zpool to Storage Manager: oxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03 file = sled-agent/src/long_running_tasks.rs:232
302023-11-20T07:50:11.917ZINFOSledAgent: Upserting synthetic zpool to Storage Manager: oxp_14b4dc87-ab46-49fb-a4b4-d361ae214c03 file = sled-agent/src/long_running_tasks.rs:232
312023-11-20T07:50:11.917ZINFOSledAgent: Upserting synthetic zpool to Storage Manager: oxp_24b4dc87-ab46-49fb-a4b4-d361ae214c03 file = sled-agent/src/long_running_tasks.rs:232
322023-11-20T07:50:11.917ZINFOSledAgent: Upserting synthetic zpool to Storage Manager: oxp_cd70d7f6-2354-4bf2-8012-55bf9eaf7930 file = sled-agent/src/long_running_tasks.rs:232
332023-11-20T07:50:11.917ZINFOSledAgent: Upserting synthetic zpool to Storage Manager: oxp_ceb4461c-cf56-4719-ad3c-14430bfdfb60 file = sled-agent/src/long_running_tasks.rs:232
342023-11-20T07:50:11.917ZINFOSledAgent: Upserting synthetic zpool to Storage Manager: oxp_31bd71cd-4736-4a12-a387-9b74b050396f file = sled-agent/src/long_running_tasks.rs:232
352023-11-20T07:50:11.917ZINFOSledAgent: Upserting synthetic zpool to Storage Manager: oxp_616b26df-e62a-4c68-b506-f4a923d8aaf7 file = sled-agent/src/long_running_tasks.rs:232
362023-11-20T07:50:11.917ZINFOSledAgent: Waiting for boot disk file = sled-agent/src/long_running_tasks.rs:88
372023-11-20T07:50:11.917ZINFOSledAgent (HardwareMonitor): Checking current full hardware snapshot file = sled-agent/src/hardware_monitor.rs:241 underlay_network_info = None
382023-11-20T07:50:11.917ZINFOSledAgent (StorageManager): Received AddDisk(Synthetic(SyntheticDisk { identity: DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-a462a7f7-b628-40fe-80ff-4e4189e2d62b", model: "synthetic-model" }, zpool_name: ZpoolName { id: a462a7f7-b628-40fe-80ff-4e4189e2d62b, kind: Internal } })) file = sled-storage/src/manager.rs:346
392023-11-20T07:50:12.101ZINFOSledAgent (StorageManager): Channel capacity decreased current = 245 file = sled-storage/src/manager.rs:326 previous = 256
402023-11-20T07:50:12.103ZINFOSledAgent: Found boot disk DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-a462a7f7-b628-40fe-80ff-4e4189e2d62b", model: "synthetic-model" } file = sled-agent/src/long_running_tasks.rs:90
412023-11-20T07:50:12.103ZINFOSledAgent (StorageManager): Received AddDisk(Synthetic(SyntheticDisk { identity: DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-b462a7f7-b628-40fe-80ff-4e4189e2d62b", model: "synthetic-model" }, zpool_name: ZpoolName { id: b462a7f7-b628-40fe-80ff-4e4189e2d62b, kind: Internal } })) file = sled-storage/src/manager.rs:346
422023-11-20T07:50:12.278ZINFOSledAgent (StorageManager): Received AddDisk(Synthetic(SyntheticDisk { identity: DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-d462a7f7-b628-40fe-80ff-4e4189e2d62b", model: "synthetic-model" }, zpool_name: ZpoolName { id: d462a7f7-b628-40fe-80ff-4e4189e2d62b, kind: External } })) file = sled-storage/src/manager.rs:346
432023-11-20T07:50:12.278ZINFOSledAgent (StorageManager): Received AddDisk(Synthetic(SyntheticDisk { identity: DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-e4b4dc87-ab46-49fb-a4b4-d361ae214c03", model: "synthetic-model" }, zpool_name: ZpoolName { id: e4b4dc87-ab46-49fb-a4b4-d361ae214c03, kind: External } })) file = sled-storage/src/manager.rs:346
442023-11-20T07:50:12.278ZINFOSledAgent (StorageManager): Received AddDisk(Synthetic(SyntheticDisk { identity: DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-f4b4dc87-ab46-49fb-a4b4-d361ae214c03", model: "synthetic-model" }, zpool_name: ZpoolName { id: f4b4dc87-ab46-49fb-a4b4-d361ae214c03, kind: External } })) file = sled-storage/src/manager.rs:346
452023-11-20T07:50:12.278ZINFOSledAgent (StorageManager): Received AddDisk(Synthetic(SyntheticDisk { identity: DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-14b4dc87-ab46-49fb-a4b4-d361ae214c03", model: "synthetic-model" }, zpool_name: ZpoolName { id: 14b4dc87-ab46-49fb-a4b4-d361ae214c03, kind: External } })) file = sled-storage/src/manager.rs:346
462023-11-20T07:50:12.278ZINFOSledAgent (StorageManager): Received AddDisk(Synthetic(SyntheticDisk { identity: DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-24b4dc87-ab46-49fb-a4b4-d361ae214c03", model: "synthetic-model" }, zpool_name: ZpoolName { id: 24b4dc87-ab46-49fb-a4b4-d361ae214c03, kind: External } })) file = sled-storage/src/manager.rs:346
472023-11-20T07:50:12.278ZINFOSledAgent (StorageManager): Received AddDisk(Synthetic(SyntheticDisk { identity: DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-cd70d7f6-2354-4bf2-8012-55bf9eaf7930", model: "synthetic-model" }, zpool_name: ZpoolName { id: cd70d7f6-2354-4bf2-8012-55bf9eaf7930, kind: External } })) file = sled-storage/src/manager.rs:346
482023-11-20T07:50:12.278ZINFOSledAgent (StorageManager): Received AddDisk(Synthetic(SyntheticDisk { identity: DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-ceb4461c-cf56-4719-ad3c-14430bfdfb60", model: "synthetic-model" }, zpool_name: ZpoolName { id: ceb4461c-cf56-4719-ad3c-14430bfdfb60, kind: External } })) file = sled-storage/src/manager.rs:346
492023-11-20T07:50:12.278ZINFOSledAgent (StorageManager): Received AddDisk(Synthetic(SyntheticDisk { identity: DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-31bd71cd-4736-4a12-a387-9b74b050396f", model: "synthetic-model" }, zpool_name: ZpoolName { id: 31bd71cd-4736-4a12-a387-9b74b050396f, kind: External } })) file = sled-storage/src/manager.rs:346
502023-11-20T07:50:12.278ZINFOSledAgent (StorageManager): Received AddDisk(Synthetic(SyntheticDisk { identity: DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-616b26df-e62a-4c68-b506-f4a923d8aaf7", model: "synthetic-model" }, zpool_name: ZpoolName { id: 616b26df-e62a-4c68-b506-f4a923d8aaf7, kind: External } })) file = sled-storage/src/manager.rs:346
512023-11-20T07:50:12.278ZINFOSledAgent (StorageManager): Received DisksChanged({}) file = sled-storage/src/manager.rs:346
522023-11-20T07:50:12.278ZINFOSledAgent (StorageManager): Channel capacity increased current = 255 file = sled-storage/src/manager.rs:336 previous = 245
532023-11-20T07:50:12.278ZINFOSledAgent (StorageManager): Received GetLatestResources(Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) }) file = sled-storage/src/manager.rs:346
542023-11-20T07:50:12.278ZINFOSledAgent (StorageMonitor): Received storage manager update file = sled-agent/src/storage_monitor.rs:111 resources = StorageResources { disks: {DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-a462a7f7-b628-40fe-80ff-4e4189e2d62b", model: "synthetic-model" }: (Synthetic(SyntheticDisk { identity: DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-a462a7f7-b628-40fe-80ff-4e4189e2d62b", model: "synthetic-model" }, zpool_name: ZpoolName { id: a462a7f7-b628-40fe-80ff-4e4189e2d62b, kind: Internal } }), Pool { name: ZpoolName { id: a462a7f7-b628-40fe-80ff-4e4189e2d62b, kind: Internal }, info: ZpoolInfo { name: "oxi_a462a7f7-b628-40fe-80ff-4e4189e2d62b", size: 20937965568, allocated: 497664, free: 20937467904, health: Online }, parent: DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-a462a7f7-b628-40fe-80ff-4e4189e2d62b", model: "synthetic-model" } }), DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-b462a7f7-b628-40fe-80ff-4e4189e2d62b", model: "synthetic-model" }: (Synthetic(SyntheticDisk { identity: DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-b462a7f7-b628-40fe-80ff-4e4189e2d62b", model: "synthetic-model" }, zpool_name: ZpoolName { id: b462a7f7-b628-40fe-80ff-4e4189e2d62b, kind: Internal } }), Pool { name: ZpoolName { id: b462a7f7-b628-40fe-80ff-4e4189e2d62b, kind: Internal }, info: ZpoolInfo { name: "oxi_b462a7f7-b628-40fe-80ff-4e4189e2d62b", size: 20937965568, allocated: 497664, free: 20937467904, health: Online }, parent: DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-b462a7f7-b628-40fe-80ff-4e4189e2d62b", model: "synthetic-model" } })} }
552023-11-20T07:50:12.278ZINFOSledAgent: Starting Bootstore file = sled-agent/src/long_running_tasks.rs:200
562023-11-20T07:50:12.278ZWARNSledAgent (bootstore): No ledger in /pool/int/a462a7f7-b628-40fe-80ff-4e4189e2d62b/cluster/bootstore-fsm-state.json file = common/src/ledger.rs:186 peer_id = pc-57495fb0328c443a91abff25dfea8602-i86pc
572023-11-20T07:50:12.278ZERROSledAgent (bootstore): Failed to read ledger: Not found in storage file = common/src/ledger.rs:90 path = /pool/int/a462a7f7-b628-40fe-80ff-4e4189e2d62b/cluster/bootstore-fsm-state.json peer_id = pc-57495fb0328c443a91abff25dfea8602-i86pc
582023-11-20T07:50:12.278ZWARNSledAgent (bootstore): No ledger in /pool/int/b462a7f7-b628-40fe-80ff-4e4189e2d62b/cluster/bootstore-fsm-state.json file = common/src/ledger.rs:186 peer_id = pc-57495fb0328c443a91abff25dfea8602-i86pc
592023-11-20T07:50:12.278ZERROSledAgent (bootstore): Failed to read ledger: Not found in storage file = common/src/ledger.rs:90 path = /pool/int/b462a7f7-b628-40fe-80ff-4e4189e2d62b/cluster/bootstore-fsm-state.json peer_id = pc-57495fb0328c443a91abff25dfea8602-i86pc
602023-11-20T07:50:12.278ZINFOSledAgent (bootstore): No ledger found. Loading Fsm::State as Uninitialized file = bootstore/src/schemes/v0/storage.rs:83 peer_id = pc-57495fb0328c443a91abff25dfea8602-i86pc
612023-11-20T07:50:12.278ZWARNSledAgent (bootstore): No ledger in /pool/int/a462a7f7-b628-40fe-80ff-4e4189e2d62b/cluster/bootstore-network-config.json file = common/src/ledger.rs:186 peer_id = pc-57495fb0328c443a91abff25dfea8602-i86pc
622023-11-20T07:50:12.278ZERROSledAgent (bootstore): Failed to read ledger: Not found in storage file = common/src/ledger.rs:90 path = /pool/int/a462a7f7-b628-40fe-80ff-4e4189e2d62b/cluster/bootstore-network-config.json peer_id = pc-57495fb0328c443a91abff25dfea8602-i86pc
632023-11-20T07:50:12.278ZWARNSledAgent (bootstore): No ledger in /pool/int/b462a7f7-b628-40fe-80ff-4e4189e2d62b/cluster/bootstore-network-config.json file = common/src/ledger.rs:186 peer_id = pc-57495fb0328c443a91abff25dfea8602-i86pc
642023-11-20T07:50:12.278ZERROSledAgent (bootstore): Failed to read ledger: Not found in storage file = common/src/ledger.rs:90 path = /pool/int/b462a7f7-b628-40fe-80ff-4e4189e2d62b/cluster/bootstore-network-config.json peer_id = pc-57495fb0328c443a91abff25dfea8602-i86pc
652023-11-20T07:50:12.278ZINFOSledAgent (bootstore): No ledger found for network config file = bootstore/src/schemes/v0/storage.rs:146 peer_id = pc-57495fb0328c443a91abff25dfea8602-i86pc
662023-11-20T07:50:12.278ZINFOSledAgent: Starting Bootstore DDMD poller file = sled-agent/src/long_running_tasks.rs:205
672023-11-20T07:50:12.278ZINFOSledAgent: Starting ZoneBundler related tasks file = sled-agent/src/long_running_tasks.rs:220
682023-11-20T07:50:12.293ZINFOSledAgent (ServiceManager): Creating ServiceManager file = sled-agent/src/services.rs:425
692023-11-20T07:50:12.293ZINFOSledAgent (auto-cleanup-task): top of bundle cleanup loop file = sled-agent/src/zone_bundle.rs:288 next_cleanup = Instant { tv_sec: 4754, tv_nsec: 675995597 } time_to_next_cleanup = 599.999934728s
702023-11-20T07:50:12.293ZINFOSledAgent (ServiceManager): Ensuring scrimlet services (enabling services) file = sled-agent/src/services.rs:2520
712023-11-20T07:50:12.293ZINFOSledAgent (StorageManager): Received GetLatestResources(Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) }) file = sled-storage/src/manager.rs:346
722023-11-20T07:50:12.293ZWARNSledAgent (BootstrapAgentStartup): No ledger in /pool/int/a462a7f7-b628-40fe-80ff-4e4189e2d62b/config/sled-agent-request.json file = common/src/ledger.rs:186
732023-11-20T07:50:12.293ZERROSledAgent (BootstrapAgentStartup): Failed to read ledger: Not found in storage file = common/src/ledger.rs:90 path = /pool/int/a462a7f7-b628-40fe-80ff-4e4189e2d62b/config/sled-agent-request.json
742023-11-20T07:50:12.293ZWARNSledAgent (BootstrapAgentStartup): No ledger in /pool/int/b462a7f7-b628-40fe-80ff-4e4189e2d62b/config/sled-agent-request.json file = common/src/ledger.rs:186
752023-11-20T07:50:12.293ZERROSledAgent (BootstrapAgentStartup): Failed to read ledger: Not found in storage file = common/src/ledger.rs:90 path = /pool/int/b462a7f7-b628-40fe-80ff-4e4189e2d62b/config/sled-agent-request.json
762023-11-20T07:50:12.296ZINFOSledAgent (dropshot (BootstrapAgent)): listening file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:195 local_addr = [fdb0:18c0:4d0d:a02a::1]:80
772023-11-20T07:50:12.296ZINFOSledAgent (SledAgentSprocketsServer): Started listening file = sled-agent/src/bootstrap/sprockets_server.rs:44 local_addr = [fdb0:18c0:4d0d:a02a::1]:12346
782023-11-20T07:50:12.297ZINFOSledAgent (RSS): Injecting RSS configuration: RackInitializeRequest { rack_subnet: fd00:1122:3344:100::, trust_quorum_peers: None, bootstrap_discovery: OnlyOurs, ntp_servers: [ "0.pool.ntp.org", ], dns_servers: [ 1.1.1.1, 9.9.9.9, ], internal_services_ip_pool_ranges: [ V4( Ipv4Range { first: 10.151.2.164, last: 10.151.2.173, }, ), ], external_dns_ips: [ 10.151.2.164, 10.151.2.165, ], external_dns_zone_name: "oxide.test", external_certificates: "<redacted>", recovery_silo: RecoverySiloConfig { silo_name: Name( "recovery", ), user_name: UserId( "recovery", ), user_password_hash: NewPasswordHash( PasswordHashString { string: "$argon2id$v=19$m=98304,t=13,p=1$RUlWc0ZxaHo0WFdrN0N6ZQ$S8p52j85GPvMhR/ek3GL0el/oProgTwWpHJZ8lsQQoY", encoding: B64, }, ), }, rack_network_config: Some( RackNetworkConfigV1 { rack_subnet: Ipv6Network { addr: fd00:1122:3344:1::, prefix: 56, }, infra_ip_first: 10.151.2.174, infra_ip_last: 10.151.2.174, ports: [ PortConfigV1 { routes: [ RouteConfig { destination: V4( Ipv4Network { addr: 0.0.0.0, prefix: 0, }, ), nexthop: 10.151.2.100, }, ], addresses: [ V4( Ipv4Network { addr: 10.151.2.174, prefix: 32, }, ), ], switch: Switch0, port: "qsfp0", uplink_port_speed: Speed40G, uplink_port_fec: None, bgp_peers: [], }, ], bgp: [], }, ), } file = sled-agent/src/rack_setup/service.rs:781
792023-11-20T07:50:12.297ZINFOSledAgent (DnsResolver): new DNS resolver addresses = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53, [fd00:1122:3344:4::1]:53, [fd00:1122:3344:5::1]:53] file = internal-dns/src/resolver.rs:60
802023-11-20T07:50:12.297ZINFOSledAgent (StorageManager): Received GetLatestResources(Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) }) file = sled-storage/src/manager.rs:346
812023-11-20T07:50:12.297ZWARNSledAgent (RSS): No ledger in /pool/int/a462a7f7-b628-40fe-80ff-4e4189e2d62b/config/rss-plan-completed.marker file = common/src/ledger.rs:186
822023-11-20T07:50:12.297ZERROSledAgent (RSS): Failed to read ledger: Not found in storage file = common/src/ledger.rs:90 path = /pool/int/a462a7f7-b628-40fe-80ff-4e4189e2d62b/config/rss-plan-completed.marker
832023-11-20T07:50:12.297ZWARNSledAgent (RSS): No ledger in /pool/int/b462a7f7-b628-40fe-80ff-4e4189e2d62b/config/rss-plan-completed.marker file = common/src/ledger.rs:186
842023-11-20T07:50:12.297ZERROSledAgent (RSS): Failed to read ledger: Not found in storage file = common/src/ledger.rs:90 path = /pool/int/b462a7f7-b628-40fe-80ff-4e4189e2d62b/config/rss-plan-completed.marker
852023-11-20T07:50:12.297ZINFOSledAgent (RSS): RSS configuration has not been fully applied yet file = sled-agent/src/rack_setup/service.rs:841
862023-11-20T07:50:12.297ZINFOSledAgent (StorageManager): Received GetLatestResources(Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) }) file = sled-storage/src/manager.rs:346
872023-11-20T07:50:12.297ZWARNSledAgent (RSS): No ledger in /pool/int/a462a7f7-b628-40fe-80ff-4e4189e2d62b/config/rss-sled-plan.json file = common/src/ledger.rs:186
882023-11-20T07:50:12.297ZERROSledAgent (RSS): Failed to read ledger: Not found in storage file = common/src/ledger.rs:90 path = /pool/int/a462a7f7-b628-40fe-80ff-4e4189e2d62b/config/rss-sled-plan.json
892023-11-20T07:50:12.297ZWARNSledAgent (RSS): No ledger in /pool/int/b462a7f7-b628-40fe-80ff-4e4189e2d62b/config/rss-sled-plan.json file = common/src/ledger.rs:186
902023-11-20T07:50:12.297ZERROSledAgent (RSS): Failed to read ledger: Not found in storage file = common/src/ledger.rs:90 path = /pool/int/b462a7f7-b628-40fe-80ff-4e4189e2d62b/config/rss-sled-plan.json
912023-11-20T07:50:12.297ZINFOSledAgent (RSS): Creating new allocation plan file = sled-agent/src/rack_setup/service.rs:882
922023-11-20T07:50:12.297ZINFOSledAgent (RSS): Serializing plan file = sled-agent/src/rack_setup/plan/sled.rs:118
932023-11-20T07:50:12.297ZINFOSledAgent (RSS): Creating plan for the sled at fdb0:18c0:4d0d:a02a::1 file = sled-agent/src/rack_setup/plan/sled.rs:91
942023-11-20T07:50:12.297ZINFOSledAgent (StorageManager): Received GetLatestResources(Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) }) file = sled-storage/src/manager.rs:346
952023-11-20T07:50:12.297ZINFOSledAgent (RSS): Writing ledger to /pool/int/a462a7f7-b628-40fe-80ff-4e4189e2d62b/config/.rss-sled-plan.json.tmp file = common/src/ledger.rs:197
962023-11-20T07:50:12.298ZINFOSledAgent (RSS): Writing ledger to /pool/int/b462a7f7-b628-40fe-80ff-4e4189e2d62b/config/.rss-sled-plan.json.tmp file = common/src/ledger.rs:197
972023-11-20T07:50:12.298ZINFOSledAgent (RSS): Sled plan written to storage file = sled-agent/src/rack_setup/plan/sled.rs:138
982023-11-20T07:50:12.298ZINFOSledAgent (RSS): Writing Rack Network Configuration to bootstore file = sled-agent/src/rack_setup/service.rs:915
992023-11-20T07:50:12.298ZINFOSledAgent (bootstore): Attempting to update network config with generation: 1, current_generation: 0 file = bootstore/src/schemes/v0/peer.rs:556 peer_id = pc-57495fb0328c443a91abff25dfea8602-i86pc
1002023-11-20T07:50:12.298ZINFOSledAgent (bootstore): Writing ledger to /pool/int/a462a7f7-b628-40fe-80ff-4e4189e2d62b/cluster/.bootstore-network-config.json.tmp file = common/src/ledger.rs:197 peer_id = pc-57495fb0328c443a91abff25dfea8602-i86pc
1012023-11-20T07:50:12.298ZINFOSledAgent (bootstore): Writing ledger to /pool/int/b462a7f7-b628-40fe-80ff-4e4189e2d62b/cluster/.bootstore-network-config.json.tmp file = common/src/ledger.rs:197 peer_id = pc-57495fb0328c443a91abff25dfea8602-i86pc
1022023-11-20T07:50:12.298ZINFOSledAgent (bootstore): Broadcasting network config with generation 1 file = bootstore/src/schemes/v0/peer.rs:640 peer_id = pc-57495fb0328c443a91abff25dfea8602-i86pc
1032023-11-20T07:50:12.298ZINFOSledAgent (BootstrapAgentRssHandler): Received initialization request from RSS file = sled-agent/src/bootstrap/rss_handle.rs:217 request = StartSledAgentRequest { generation: 0, schema_version: 1, body: StartSledAgentRequestBody { id: e75545d5-2741-43ba-8c56-e7f2943a5b82, rack_id: e54261f8-48ed-419b-92cd-78ba43007b3f, use_trust_quorum: false, is_lrtq_learner: false, subnet: Ipv6Subnet { net: Ipv6Net(Ipv6Network { addr: fd00:1122:3344:101::, prefix: 64 }) } } } target_sled = [fdb0:18c0:4d0d:a02a::1]:12346
1042023-11-20T07:50:12.298ZINFOSledAgent (SledAgentSprocketsServer): Accepted connection file = sled-agent/src/bootstrap/sprockets_server.rs:66 remote_addr = [fdb0:18c0:4d0d:a02a::1]:46844
1052023-11-20T07:50:12.298ZINFOSledAgent (SledAgentMain): Loading Sled Agent: StartSledAgentRequest { generation: 0, schema_version: 1, body: StartSledAgentRequestBody { id: e75545d5-2741-43ba-8c56-e7f2943a5b82, rack_id: e54261f8-48ed-419b-92cd-78ba43007b3f, use_trust_quorum: false, is_lrtq_learner: false, subnet: Ipv6Subnet { net: Ipv6Net(Ipv6Network { addr: fd00:1122:3344:101::, prefix: 64 }) } } } file = sled-agent/src/bootstrap/server.rs:364
1062023-11-20T07:50:12.298ZINFOSledAgent (SledAgentMain): KeyManager: using hardcoded secret retriever file = sled-agent/src/bootstrap/server.rs:379
1072023-11-20T07:50:12.298ZINFOSledAgent: setting up sled agent server file = sled-agent/src/server.rs:47
1082023-11-20T07:50:12.298ZINFOSledAgent (StorageManager): Received KeyManagerReady file = sled-storage/src/manager.rs:346
1092023-11-20T07:50:12.298ZINFOSledAgent (StorageManager): Attempting to add queued disks file = sled-storage/src/manager.rs:394 num_disks = 9
1102023-11-20T07:50:12.299ZINFOSledAgent (DnsResolver): new DNS resolver addresses = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53, [fd00:1122:3344:4::1]:53, [fd00:1122:3344:5::1]:53] file = internal-dns/src/resolver.rs:60
1112023-11-20T07:50:12.302ZINFOSledAgent (ServiceManager): Enabling switch zone (new) file = sled-agent/src/services.rs:2765
1122023-11-20T07:50:12.307ZINFOSledAgent (StorageManager): Loading latest secret disk_id = DiskIdentity {\n vendor: "synthetic-vendor",\n serial: "synthetic-serial-e4b4dc87-ab46-49fb-a4b4-d361ae214c03",\n model: "synthetic-model",\n} file = sled-storage/src/dataset.rs:261
1132023-11-20T07:50:12.308ZINFOSledAgent (StorageManager): Loaded latest secret disk_id = DiskIdentity {\n vendor: "synthetic-vendor",\n serial: "synthetic-serial-e4b4dc87-ab46-49fb-a4b4-d361ae214c03",\n model: "synthetic-model",\n} epoch = 0 file = sled-storage/src/dataset.rs:263
1142023-11-20T07:50:12.308ZINFOSledAgent (StorageManager): Retrieving key disk_id = DiskIdentity {\n vendor: "synthetic-vendor",\n serial: "synthetic-serial-e4b4dc87-ab46-49fb-a4b4-d361ae214c03",\n model: "synthetic-model",\n} epoch = 0 file = sled-storage/src/dataset.rs:267
1152023-11-20T07:50:12.308ZINFOSledAgent (StorageManager): Got key disk_id = DiskIdentity {\n vendor: "synthetic-vendor",\n serial: "synthetic-serial-e4b4dc87-ab46-49fb-a4b4-d361ae214c03",\n model: "synthetic-model",\n} epoch = 0 file = sled-storage/src/dataset.rs:269
1162023-11-20T07:50:12.308ZINFOSledAgent (StorageManager): Created keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-e4b4dc87-ab46-49fb-a4b4-d361ae214c03-synthetic-model-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:37
1172023-11-20T07:50:12.308ZINFOSledAgent (StorageManager): Ensuring encrypted filesystem: crypt for epoch 0 file = sled-storage/src/dataset.rs:281
1182023-11-20T07:50:12.339ZINFOSledAgent (StorageManager): Zeroed and unlinked keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-e4b4dc87-ab46-49fb-a4b4-d361ae214c03-synthetic-model-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:56
1192023-11-20T07:50:12.346ZINFOSledAgent (StorageManager): Automatically destroying dataset: oxp_e4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone file = sled-storage/src/dataset.rs:323
1202023-11-20T07:50:12.369ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {}
1212023-11-20T07:50:12.383ZINFOSledAgent: SledAgent::new(..) starting file = sled-agent/src/sled_agent.rs:303 sled_id = e75545d5-2741-43ba-8c56-e7f2943a5b82
1222023-11-20T07:50:12.383ZINFOSledAgent (BootstrapAgentStartup): Sending prefix to ddmd for advertisement DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:91 prefix = Ipv6Prefix { addr: fd00:1122:3344:101::, len: 64 }
1232023-11-20T07:50:12.468ZINFOSledAgent (StorageManager): Loading latest secret disk_id = DiskIdentity {\n vendor: "synthetic-vendor",\n serial: "synthetic-serial-f4b4dc87-ab46-49fb-a4b4-d361ae214c03",\n model: "synthetic-model",\n} file = sled-storage/src/dataset.rs:261
1242023-11-20T07:50:12.468ZINFOSledAgent (StorageManager): Loaded latest secret disk_id = DiskIdentity {\n vendor: "synthetic-vendor",\n serial: "synthetic-serial-f4b4dc87-ab46-49fb-a4b4-d361ae214c03",\n model: "synthetic-model",\n} epoch = 0 file = sled-storage/src/dataset.rs:263
1252023-11-20T07:50:12.468ZINFOSledAgent (StorageManager): Retrieving key disk_id = DiskIdentity {\n vendor: "synthetic-vendor",\n serial: "synthetic-serial-f4b4dc87-ab46-49fb-a4b4-d361ae214c03",\n model: "synthetic-model",\n} epoch = 0 file = sled-storage/src/dataset.rs:267
1262023-11-20T07:50:12.468ZINFOSledAgent (StorageManager): Got key disk_id = DiskIdentity {\n vendor: "synthetic-vendor",\n serial: "synthetic-serial-f4b4dc87-ab46-49fb-a4b4-d361ae214c03",\n model: "synthetic-model",\n} epoch = 0 file = sled-storage/src/dataset.rs:269
1272023-11-20T07:50:12.468ZINFOSledAgent (StorageManager): Created keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-f4b4dc87-ab46-49fb-a4b4-d361ae214c03-synthetic-model-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:37
1282023-11-20T07:50:12.468ZINFOSledAgent (StorageManager): Ensuring encrypted filesystem: crypt for epoch 0 file = sled-storage/src/dataset.rs:281
1292023-11-20T07:50:12.495ZINFOSledAgent (StorageManager): Zeroed and unlinked keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-f4b4dc87-ab46-49fb-a4b4-d361ae214c03-synthetic-model-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:56
1302023-11-20T07:50:12.503ZINFOSledAgent (StorageManager): Automatically destroying dataset: oxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone file = sled-storage/src/dataset.rs:323
1312023-11-20T07:50:12.621ZINFOSledAgent (StorageManager): Loading latest secret disk_id = DiskIdentity {\n vendor: "synthetic-vendor",\n serial: "synthetic-serial-24b4dc87-ab46-49fb-a4b4-d361ae214c03",\n model: "synthetic-model",\n} file = sled-storage/src/dataset.rs:261
1322023-11-20T07:50:12.621ZINFOSledAgent (StorageManager): Loaded latest secret disk_id = DiskIdentity {\n vendor: "synthetic-vendor",\n serial: "synthetic-serial-24b4dc87-ab46-49fb-a4b4-d361ae214c03",\n model: "synthetic-model",\n} epoch = 0 file = sled-storage/src/dataset.rs:263
1332023-11-20T07:50:12.621ZINFOSledAgent (StorageManager): Retrieving key disk_id = DiskIdentity {\n vendor: "synthetic-vendor",\n serial: "synthetic-serial-24b4dc87-ab46-49fb-a4b4-d361ae214c03",\n model: "synthetic-model",\n} epoch = 0 file = sled-storage/src/dataset.rs:267
1342023-11-20T07:50:12.621ZINFOSledAgent (StorageManager): Got key disk_id = DiskIdentity {\n vendor: "synthetic-vendor",\n serial: "synthetic-serial-24b4dc87-ab46-49fb-a4b4-d361ae214c03",\n model: "synthetic-model",\n} epoch = 0 file = sled-storage/src/dataset.rs:269
1352023-11-20T07:50:12.621ZINFOSledAgent (StorageManager): Created keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-24b4dc87-ab46-49fb-a4b4-d361ae214c03-synthetic-model-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:37
1362023-11-20T07:50:12.621ZINFOSledAgent (StorageManager): Ensuring encrypted filesystem: crypt for epoch 0 file = sled-storage/src/dataset.rs:281
1372023-11-20T07:50:12.648ZINFOSledAgent (StorageManager): Zeroed and unlinked keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-24b4dc87-ab46-49fb-a4b4-d361ae214c03-synthetic-model-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:56
1382023-11-20T07:50:12.656ZINFOSledAgent (StorageManager): Automatically destroying dataset: oxp_24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone file = sled-storage/src/dataset.rs:323
1392023-11-20T07:50:12.778ZINFOSledAgent (StorageManager): Loading latest secret disk_id = DiskIdentity {\n vendor: "synthetic-vendor",\n serial: "synthetic-serial-14b4dc87-ab46-49fb-a4b4-d361ae214c03",\n model: "synthetic-model",\n} file = sled-storage/src/dataset.rs:261
1402023-11-20T07:50:12.778ZINFOSledAgent (StorageManager): Loaded latest secret disk_id = DiskIdentity {\n vendor: "synthetic-vendor",\n serial: "synthetic-serial-14b4dc87-ab46-49fb-a4b4-d361ae214c03",\n model: "synthetic-model",\n} epoch = 0 file = sled-storage/src/dataset.rs:263
1412023-11-20T07:50:12.778ZINFOSledAgent (StorageManager): Retrieving key disk_id = DiskIdentity {\n vendor: "synthetic-vendor",\n serial: "synthetic-serial-14b4dc87-ab46-49fb-a4b4-d361ae214c03",\n model: "synthetic-model",\n} epoch = 0 file = sled-storage/src/dataset.rs:267
1422023-11-20T07:50:12.778ZINFOSledAgent (StorageManager): Got key disk_id = DiskIdentity {\n vendor: "synthetic-vendor",\n serial: "synthetic-serial-14b4dc87-ab46-49fb-a4b4-d361ae214c03",\n model: "synthetic-model",\n} epoch = 0 file = sled-storage/src/dataset.rs:269
1432023-11-20T07:50:12.778ZINFOSledAgent (StorageManager): Created keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-14b4dc87-ab46-49fb-a4b4-d361ae214c03-synthetic-model-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:37
1442023-11-20T07:50:12.778ZINFOSledAgent (StorageManager): Ensuring encrypted filesystem: crypt for epoch 0 file = sled-storage/src/dataset.rs:281
1452023-11-20T07:50:12.804ZINFOSledAgent (StorageManager): Zeroed and unlinked keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-14b4dc87-ab46-49fb-a4b4-d361ae214c03-synthetic-model-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:56
1462023-11-20T07:50:12.812ZINFOSledAgent (StorageManager): Automatically destroying dataset: oxp_14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone file = sled-storage/src/dataset.rs:323
1472023-11-20T07:50:12.930ZINFOSledAgent (StorageManager): Loading latest secret disk_id = DiskIdentity {\n vendor: "synthetic-vendor",\n serial: "synthetic-serial-616b26df-e62a-4c68-b506-f4a923d8aaf7",\n model: "synthetic-model",\n} file = sled-storage/src/dataset.rs:261
1482023-11-20T07:50:12.931ZINFOSledAgent (StorageManager): Loaded latest secret disk_id = DiskIdentity {\n vendor: "synthetic-vendor",\n serial: "synthetic-serial-616b26df-e62a-4c68-b506-f4a923d8aaf7",\n model: "synthetic-model",\n} epoch = 0 file = sled-storage/src/dataset.rs:263
1492023-11-20T07:50:12.931ZINFOSledAgent (StorageManager): Retrieving key disk_id = DiskIdentity {\n vendor: "synthetic-vendor",\n serial: "synthetic-serial-616b26df-e62a-4c68-b506-f4a923d8aaf7",\n model: "synthetic-model",\n} epoch = 0 file = sled-storage/src/dataset.rs:267
1502023-11-20T07:50:12.931ZINFOSledAgent (StorageManager): Got key disk_id = DiskIdentity {\n vendor: "synthetic-vendor",\n serial: "synthetic-serial-616b26df-e62a-4c68-b506-f4a923d8aaf7",\n model: "synthetic-model",\n} epoch = 0 file = sled-storage/src/dataset.rs:269
1512023-11-20T07:50:12.931ZINFOSledAgent (StorageManager): Created keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-616b26df-e62a-4c68-b506-f4a923d8aaf7-synthetic-model-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:37
1522023-11-20T07:50:12.931ZINFOSledAgent (StorageManager): Ensuring encrypted filesystem: crypt for epoch 0 file = sled-storage/src/dataset.rs:281
1532023-11-20T07:50:12.958ZINFOSledAgent (StorageManager): Zeroed and unlinked keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-616b26df-e62a-4c68-b506-f4a923d8aaf7-synthetic-model-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:56
1542023-11-20T07:50:12.966ZINFOSledAgent (StorageManager): Automatically destroying dataset: oxp_616b26df-e62a-4c68-b506-f4a923d8aaf7/crypt/zone file = sled-storage/src/dataset.rs:323
1552023-11-20T07:50:13.087ZINFOSledAgent (StorageManager): Loading latest secret disk_id = DiskIdentity {\n vendor: "synthetic-vendor",\n serial: "synthetic-serial-31bd71cd-4736-4a12-a387-9b74b050396f",\n model: "synthetic-model",\n} file = sled-storage/src/dataset.rs:261
1562023-11-20T07:50:13.087ZINFOSledAgent (StorageManager): Loaded latest secret disk_id = DiskIdentity {\n vendor: "synthetic-vendor",\n serial: "synthetic-serial-31bd71cd-4736-4a12-a387-9b74b050396f",\n model: "synthetic-model",\n} epoch = 0 file = sled-storage/src/dataset.rs:263
1572023-11-20T07:50:13.087ZINFOSledAgent (StorageManager): Retrieving key disk_id = DiskIdentity {\n vendor: "synthetic-vendor",\n serial: "synthetic-serial-31bd71cd-4736-4a12-a387-9b74b050396f",\n model: "synthetic-model",\n} epoch = 0 file = sled-storage/src/dataset.rs:267
1582023-11-20T07:50:13.087ZINFOSledAgent (StorageManager): Got key disk_id = DiskIdentity {\n vendor: "synthetic-vendor",\n serial: "synthetic-serial-31bd71cd-4736-4a12-a387-9b74b050396f",\n model: "synthetic-model",\n} epoch = 0 file = sled-storage/src/dataset.rs:269
1592023-11-20T07:50:13.087ZINFOSledAgent (StorageManager): Created keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-31bd71cd-4736-4a12-a387-9b74b050396f-synthetic-model-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:37
1602023-11-20T07:50:13.087ZINFOSledAgent (StorageManager): Ensuring encrypted filesystem: crypt for epoch 0 file = sled-storage/src/dataset.rs:281
1612023-11-20T07:50:13.115ZINFOSledAgent (StorageManager): Zeroed and unlinked keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-31bd71cd-4736-4a12-a387-9b74b050396f-synthetic-model-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:56
1622023-11-20T07:50:13.123ZINFOSledAgent (StorageManager): Automatically destroying dataset: oxp_31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone file = sled-storage/src/dataset.rs:323
1632023-11-20T07:50:13.244ZINFOSledAgent (StorageManager): Loading latest secret disk_id = DiskIdentity {\n vendor: "synthetic-vendor",\n serial: "synthetic-serial-d462a7f7-b628-40fe-80ff-4e4189e2d62b",\n model: "synthetic-model",\n} file = sled-storage/src/dataset.rs:261
1642023-11-20T07:50:13.244ZINFOSledAgent (StorageManager): Loaded latest secret disk_id = DiskIdentity {\n vendor: "synthetic-vendor",\n serial: "synthetic-serial-d462a7f7-b628-40fe-80ff-4e4189e2d62b",\n model: "synthetic-model",\n} epoch = 0 file = sled-storage/src/dataset.rs:263
1652023-11-20T07:50:13.244ZINFOSledAgent (StorageManager): Retrieving key disk_id = DiskIdentity {\n vendor: "synthetic-vendor",\n serial: "synthetic-serial-d462a7f7-b628-40fe-80ff-4e4189e2d62b",\n model: "synthetic-model",\n} epoch = 0 file = sled-storage/src/dataset.rs:267
1662023-11-20T07:50:13.244ZINFOSledAgent (StorageManager): Got key disk_id = DiskIdentity {\n vendor: "synthetic-vendor",\n serial: "synthetic-serial-d462a7f7-b628-40fe-80ff-4e4189e2d62b",\n model: "synthetic-model",\n} epoch = 0 file = sled-storage/src/dataset.rs:269
1672023-11-20T07:50:13.244ZINFOSledAgent (StorageManager): Created keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-d462a7f7-b628-40fe-80ff-4e4189e2d62b-synthetic-model-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:37
1682023-11-20T07:50:13.244ZINFOSledAgent (StorageManager): Ensuring encrypted filesystem: crypt for epoch 0 file = sled-storage/src/dataset.rs:281
1692023-11-20T07:50:13.271ZINFOSledAgent (StorageManager): Zeroed and unlinked keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-d462a7f7-b628-40fe-80ff-4e4189e2d62b-synthetic-model-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:56
1702023-11-20T07:50:13.279ZINFOSledAgent (StorageManager): Automatically destroying dataset: oxp_d462a7f7-b628-40fe-80ff-4e4189e2d62b/crypt/zone file = sled-storage/src/dataset.rs:323
1712023-11-20T07:50:13.397ZINFOSledAgent (StorageManager): Loading latest secret disk_id = DiskIdentity {\n vendor: "synthetic-vendor",\n serial: "synthetic-serial-cd70d7f6-2354-4bf2-8012-55bf9eaf7930",\n model: "synthetic-model",\n} file = sled-storage/src/dataset.rs:261
1722023-11-20T07:50:13.397ZINFOSledAgent (StorageManager): Loaded latest secret disk_id = DiskIdentity {\n vendor: "synthetic-vendor",\n serial: "synthetic-serial-cd70d7f6-2354-4bf2-8012-55bf9eaf7930",\n model: "synthetic-model",\n} epoch = 0 file = sled-storage/src/dataset.rs:263
1732023-11-20T07:50:13.397ZINFOSledAgent (StorageManager): Retrieving key disk_id = DiskIdentity {\n vendor: "synthetic-vendor",\n serial: "synthetic-serial-cd70d7f6-2354-4bf2-8012-55bf9eaf7930",\n model: "synthetic-model",\n} epoch = 0 file = sled-storage/src/dataset.rs:267
1742023-11-20T07:50:13.397ZINFOSledAgent (StorageManager): Got key disk_id = DiskIdentity {\n vendor: "synthetic-vendor",\n serial: "synthetic-serial-cd70d7f6-2354-4bf2-8012-55bf9eaf7930",\n model: "synthetic-model",\n} epoch = 0 file = sled-storage/src/dataset.rs:269
1752023-11-20T07:50:13.397ZINFOSledAgent (StorageManager): Created keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-cd70d7f6-2354-4bf2-8012-55bf9eaf7930-synthetic-model-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:37
1762023-11-20T07:50:13.397ZINFOSledAgent (StorageManager): Ensuring encrypted filesystem: crypt for epoch 0 file = sled-storage/src/dataset.rs:281
1772023-11-20T07:50:13.425ZINFOSledAgent (StorageManager): Zeroed and unlinked keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-cd70d7f6-2354-4bf2-8012-55bf9eaf7930-synthetic-model-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:56
1782023-11-20T07:50:13.433ZINFOSledAgent (StorageManager): Automatically destroying dataset: oxp_cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone file = sled-storage/src/dataset.rs:323
1792023-11-20T07:50:13.551ZINFOSledAgent (StorageManager): Loading latest secret disk_id = DiskIdentity {\n vendor: "synthetic-vendor",\n serial: "synthetic-serial-ceb4461c-cf56-4719-ad3c-14430bfdfb60",\n model: "synthetic-model",\n} file = sled-storage/src/dataset.rs:261
1802023-11-20T07:50:13.551ZINFOSledAgent (StorageManager): Loaded latest secret disk_id = DiskIdentity {\n vendor: "synthetic-vendor",\n serial: "synthetic-serial-ceb4461c-cf56-4719-ad3c-14430bfdfb60",\n model: "synthetic-model",\n} epoch = 0 file = sled-storage/src/dataset.rs:263
1812023-11-20T07:50:13.551ZINFOSledAgent (StorageManager): Retrieving key disk_id = DiskIdentity {\n vendor: "synthetic-vendor",\n serial: "synthetic-serial-ceb4461c-cf56-4719-ad3c-14430bfdfb60",\n model: "synthetic-model",\n} epoch = 0 file = sled-storage/src/dataset.rs:267
1822023-11-20T07:50:13.551ZINFOSledAgent (StorageManager): Got key disk_id = DiskIdentity {\n vendor: "synthetic-vendor",\n serial: "synthetic-serial-ceb4461c-cf56-4719-ad3c-14430bfdfb60",\n model: "synthetic-model",\n} epoch = 0 file = sled-storage/src/dataset.rs:269
1832023-11-20T07:50:13.551ZINFOSledAgent (StorageManager): Created keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-ceb4461c-cf56-4719-ad3c-14430bfdfb60-synthetic-model-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:37
1842023-11-20T07:50:13.552ZINFOSledAgent (StorageManager): Ensuring encrypted filesystem: crypt for epoch 0 file = sled-storage/src/dataset.rs:281
1852023-11-20T07:50:13.579ZINFOSledAgent (StorageManager): Zeroed and unlinked keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-ceb4461c-cf56-4719-ad3c-14430bfdfb60-synthetic-model-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:56
1862023-11-20T07:50:13.586ZINFOSledAgent (StorageManager): Automatically destroying dataset: oxp_ceb4461c-cf56-4719-ad3c-14430bfdfb60/crypt/zone file = sled-storage/src/dataset.rs:323
1872023-11-20T07:50:13.696ZINFOSledAgent (StorageManager): Received GetLatestResources(Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) }) file = sled-storage/src/manager.rs:346
1882023-11-20T07:50:13.696ZINFOSledAgent (StorageManager): Received GetLatestResources(Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) }) file = sled-storage/src/manager.rs:346
1892023-11-20T07:50:13.696ZINFOSledAgent: Requested swap device of size 64 GiB file = sled-agent/src/sled_agent.rs:315 sled_id = e75545d5-2741-43ba-8c56-e7f2943a5b82
1902023-11-20T07:50:13.702ZINFOSledAgent (StorageMonitor): Received storage manager update file = sled-agent/src/storage_monitor.rs:111 resources = StorageResources { disks: {DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-14b4dc87-ab46-49fb-a4b4-d361ae214c03", model: "synthetic-model" }: (Synthetic(SyntheticDisk { identity: DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-14b4dc87-ab46-49fb-a4b4-d361ae214c03", model: "synthetic-model" }, zpool_name: ZpoolName { id: 14b4dc87-ab46-49fb-a4b4-d361ae214c03, kind: External } }), Pool { name: ZpoolName { id: 14b4dc87-ab46-49fb-a4b4-d361ae214c03, kind: External }, info: ZpoolInfo { name: "oxp_14b4dc87-ab46-49fb-a4b4-d361ae214c03", size: 20937965568, allocated: 775680, free: 20937189888, health: Online }, parent: DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-14b4dc87-ab46-49fb-a4b4-d361ae214c03", model: "synthetic-model" } }), DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-24b4dc87-ab46-49fb-a4b4-d361ae214c03", model: "synthetic-model" }: (Synthetic(SyntheticDisk { identity: DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-24b4dc87-ab46-49fb-a4b4-d361ae214c03", model: "synthetic-model" }, zpool_name: ZpoolName { id: 24b4dc87-ab46-49fb-a4b4-d361ae214c03, kind: External } }), Pool { name: ZpoolName { id: 24b4dc87-ab46-49fb-a4b4-d361ae214c03, kind: External }, info: ZpoolInfo { name: "oxp_24b4dc87-ab46-49fb-a4b4-d361ae214c03", size: 20937965568, allocated: 1213440, free: 20936752128, health: Online }, parent: DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-24b4dc87-ab46-49fb-a4b4-d361ae214c03", model: "synthetic-model" } }), DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-31bd71cd-4736-4a12-a387-9b74b050396f", model: "synthetic-model" }: (Synthetic(SyntheticDisk { identity: DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-31bd71cd-4736-4a12-a387-9b74b050396f", model: "synthetic-model" }, zpool_name: ZpoolName { id: 31bd71cd-4736-4a12-a387-9b74b050396f, kind: External } }), Pool { name: ZpoolName { id: 31bd71cd-4736-4a12-a387-9b74b050396f, kind: External }, info: ZpoolInfo { name: "oxp_31bd71cd-4736-4a12-a387-9b74b050396f", size: 20937965568, allocated: 772608, free: 20937192960, health: Online }, parent: DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-31bd71cd-4736-4a12-a387-9b74b050396f", model: "synthetic-model" } }), DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-616b26df-e62a-4c68-b506-f4a923d8aaf7", model: "synthetic-model" }: (Synthetic(SyntheticDisk { identity: DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-616b26df-e62a-4c68-b506-f4a923d8aaf7", model: "synthetic-model" }, zpool_name: ZpoolName { id: 616b26df-e62a-4c68-b506-f4a923d8aaf7, kind: External } }), Pool { name: ZpoolName { id: 616b26df-e62a-4c68-b506-f4a923d8aaf7, kind: External }, info: ZpoolInfo { name: "oxp_616b26df-e62a-4c68-b506-f4a923d8aaf7", size: 20937965568, allocated: 1728000, free: 20936237568, health: Online }, parent: DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-616b26df-e62a-4c68-b506-f4a923d8aaf7", model: "synthetic-model" } }), DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-a462a7f7-b628-40fe-80ff-4e4189e2d62b", model: "synthetic-model" }: (Synthetic(SyntheticDisk { identity: DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-a462a7f7-b628-40fe-80ff-4e4189e2d62b", model: "synthetic-model" }, zpool_name: ZpoolName { id: a462a7f7-b628-40fe-80ff-4e4189e2d62b, kind: Internal } }), Pool { name: ZpoolName { id: a462a7f7-b628-40fe-80ff-4e4189e2d62b, kind: Internal }, info: ZpoolInfo { name: "oxi_a462a7f7-b628-40fe-80ff-4e4189e2d62b", size: 20937965568, allocated: 497664, free: 20937467904, health: Online }, parent: DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-a462a7f7-b628-40fe-80ff-4e4189e2d62b", model: "synthetic-model" } }), DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-b462a7f7-b628-40fe-80ff-4e4189e2d62b", model: "synthetic-model" }: (Synthetic(SyntheticDisk { identity: DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-b462a7f7-b628-40fe-80ff-4e4189e2d62b", model: "synthetic-model" }, zpool_name: ZpoolName { id: b462a7f7-b628-40fe-80ff-4e4189e2d62b, kind: Internal } }), Pool { name: ZpoolName { id: b462a7f7-b628-40fe-80ff-4e4189e2d62b, kind: Internal }, info: ZpoolInfo { name: "oxi_b462a7f7-b628-40fe-80ff-4e4189e2d62b", size: 20937965568, allocated: 497664, free: 20937467904, health: Online }, parent: DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-b462a7f7-b628-40fe-80ff-4e4189e2d62b", model: "synthetic-model" } }), DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-cd70d7f6-2354-4bf2-8012-55bf9eaf7930", model: "synthetic-model" }: (Synthetic(SyntheticDisk { identity: DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-cd70d7f6-2354-4bf2-8012-55bf9eaf7930", model: "synthetic-model" }, zpool_name: ZpoolName { id: cd70d7f6-2354-4bf2-8012-55bf9eaf7930, kind: External } }), Pool { name: ZpoolName { id: cd70d7f6-2354-4bf2-8012-55bf9eaf7930, kind: External }, info: ZpoolInfo { name: "oxp_cd70d7f6-2354-4bf2-8012-55bf9eaf7930", size: 20937965568, allocated: 781824, free: 20937183744, health: Online }, parent: DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-cd70d7f6-2354-4bf2-8012-55bf9eaf7930", model: "synthetic-model" } }), DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-ceb4461c-cf56-4719-ad3c-14430bfdfb60", model: "synthetic-model" }: (Synthetic(SyntheticDisk { identity: DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-ceb4461c-cf56-4719-ad3c-14430bfdfb60", model: "synthetic-model" }, zpool_name: ZpoolName { id: ceb4461c-cf56-4719-ad3c-14430bfdfb60, kind: External } }), Pool { name: ZpoolName { id: ceb4461c-cf56-4719-ad3c-14430bfdfb60, kind: External }, info: ZpoolInfo { name: "oxp_ceb4461c-cf56-4719-ad3c-14430bfdfb60", size: 20937965568, allocated: 834048, free: 20937131520, health: Online }, parent: DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-ceb4461c-cf56-4719-ad3c-14430bfdfb60", model: "synthetic-model" } }), DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-d462a7f7-b628-40fe-80ff-4e4189e2d62b", model: "synthetic-model" }: (Synthetic(SyntheticDisk { identity: DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-d462a7f7-b628-40fe-80ff-4e4189e2d62b", model: "synthetic-model" }, zpool_name: ZpoolName { id: d462a7f7-b628-40fe-80ff-4e4189e2d62b, kind: External } }), Pool { name: ZpoolName { id: d462a7f7-b628-40fe-80ff-4e4189e2d62b, kind: External }, info: ZpoolInfo { name: "oxp_d462a7f7-b628-40fe-80ff-4e4189e2d62b", size: 20937965568, allocated: 772608, free: 20937192960, health: Online }, parent: DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-d462a7f7-b628-40fe-80ff-4e4189e2d62b", model: "synthetic-model" } }), DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-e4b4dc87-ab46-49fb-a4b4-d361ae214c03", model: "synthetic-model" }: (Synthetic(SyntheticDisk { identity: DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-e4b4dc87-ab46-49fb-a4b4-d361ae214c03", model: "synthetic-model" }, zpool_name: ZpoolName { id: e4b4dc87-ab46-49fb-a4b4-d361ae214c03, kind: External } }), Pool { name: ZpoolName { id: e4b4dc87-ab46-49fb-a4b4-d361ae214c03, kind: External }, info: ZpoolInfo { name: "oxp_e4b4dc87-ab46-49fb-a4b4-d361ae214c03", size: 20937965568, allocated: 775680, free: 20937189888, health: Online }, parent: DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-e4b4dc87-ab46-49fb-a4b4-d361ae214c03", model: "synthetic-model" } }), DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-f4b4dc87-ab46-49fb-a4b4-d361ae214c03", model: "synthetic-model" }: (Synthetic(SyntheticDisk { identity: DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-f4b4dc87-ab46-49fb-a4b4-d361ae214c03", model: "synthetic-model" }, zpool_name: ZpoolName { id: f4b4dc87-ab46-49fb-a4b4-d361ae214c03, kind: External } }), Pool { name: ZpoolName { id: f4b4dc87-ab46-49fb-a4b4-d361ae214c03, kind: External }, info: ZpoolInfo { name: "oxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03", size: 20937965568, allocated: 772608, free: 20937192960, health: Online }, parent: DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-f4b4dc87-ab46-49fb-a4b4-d361ae214c03", model: "synthetic-model" } })} }
1912023-11-20T07:50:13.719ZINFOSledAgent: attempting to create encrypted zvol: name="oxi_a462a7f7-b628-40fe-80ff-4e4189e2d62b/swap", size_gb=64 file = sled-agent/src/swap_device.rs:153
1922023-11-20T07:50:13.721ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_switch file = illumos-utils/src/zone.rs:314
1932023-11-20T07:50:13.739ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_switch file = illumos-utils/src/zone.rs:353
1942023-11-20T07:50:13.742ZINFOSledAgent: successfully created encrypted zvol: name="oxi_a462a7f7-b628-40fe-80ff-4e4189e2d62b/swap", size_gb=64 file = sled-agent/src/swap_device.rs:249
1952023-11-20T07:50:13.742ZINFOSledAgent: adding swap device: swapname="/dev/zvol/dsk/oxi_a462a7f7-b628-40fe-80ff-4e4189e2d62b/swap" file = sled-agent/src/swap_device.rs:96
1962023-11-20T07:50:13.747ZINFOSledAgent: Mounting backing filesystems file = sled-agent/src/sled_agent.rs:330 sled_id = e75545d5-2741-43ba-8c56-e7f2943a5b82
1972023-11-20T07:50:13.747ZINFOSledAgent (BackingFs): Processing fmd file = sled-agent/src/backing_fs.rs:126
1982023-11-20T07:50:13.747ZINFOSledAgent (BackingFs): Ensuring dataset oxi_a462a7f7-b628-40fe-80ff-4e4189e2d62b/backing/fmd file = sled-agent/src/backing_fs.rs:136
1992023-11-20T07:50:13.809ZINFOSledAgent (BackingFs): Stopping service svc:/system/fmd:default file = sled-agent/src/backing_fs.rs:170
2002023-11-20T07:50:14.027ZINFOSledAgent (BackingFs): Mounting oxi_a462a7f7-b628-40fe-80ff-4e4189e2d62b/backing/fmd on /var/fm/fmd file = sled-agent/src/backing_fs.rs:178
2012023-11-20T07:50:14.039ZINFOSledAgent (BackingFs): Ensuring directory /var/fm/fmd/rsrc file = sled-agent/src/backing_fs.rs:186
2022023-11-20T07:50:14.039ZINFOSledAgent (BackingFs): Ensuring directory /var/fm/fmd/ckpt file = sled-agent/src/backing_fs.rs:186
2032023-11-20T07:50:14.039ZINFOSledAgent (BackingFs): Ensuring directory /var/fm/fmd/xprt file = sled-agent/src/backing_fs.rs:186
2042023-11-20T07:50:14.039ZINFOSledAgent (BackingFs): Starting service svc:/system/fmd:default file = sled-agent/src/backing_fs.rs:192
2052023-11-20T07:50:14.206ZINFOSledAgent: Ensuring contract reaper thread file = illumos-utils/src/running_zone.rs:160
2062023-11-20T07:50:14.733ZINFOSledAgent: using '[AddrObject { interface: "net0", name: "ll" }, AddrObject { interface: "net1", name: "ll" }]' as data links for xde driver file = illumos-utils/src/opte/illumos.rs:80 sled_id = e75545d5-2741-43ba-8c56-e7f2943a5b82
2072023-11-20T07:50:14.733ZINFOSledAgent (StorageMonitor): Underlay Available file = sled-agent/src/storage_monitor.rs:122 sled_id = e75545d5-2741-43ba-8c56-e7f2943a5b82
2082023-11-20T07:50:14.734ZINFOSledAgent (InstanceManager): 50% of 102952026112 physical ram = 51476013056 bytes) file = sled-agent/src/instance_manager.rs:172
2092023-11-20T07:50:14.734ZINFOSledAgent (InstanceManager): Setting reservoir size to 49090 MiB bytes file = sled-agent/src/instance_manager.rs:180
2102023-11-20T07:50:14.745ZINFOSledAgent (StorageMonitor): failed to notify nexus about Put(PhysicalDiskPutRequest { model: "synthetic-model", serial: "synthetic-serial-a462a7f7-b628-40fe-80ff-4e4189e2d62b", sled_id: e75545d5-2741-43ba-8c56-e7f2943a5b82, variant: M2, vendor: "synthetic-vendor" }) err = "Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/physical-disk): error trying to connect: dns error: proto error: io error: No route to host (os error 148)" file = sled-agent/src/storage_monitor.rs:219
2112023-11-20T07:50:14.746ZINFOSledAgent (StorageMonitor): failed to notify nexus about a new pool oxp_14b4dc87-ab46-49fb-a4b4-d361ae214c03 on disk DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-14b4dc87-ab46-49fb-a4b4-d361ae214c03", model: "synthetic-model" } err = "Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/sled-agents/e75545d5-2741-43ba-8c56-e7f2943a5b82/zpools/14b4dc87-ab46-49fb-a4b4-d361ae214c03): error trying to connect: dns error: proto error: io error: No route to host (os error 148)" file = sled-agent/src/storage_monitor.rs:270
2122023-11-20T07:50:14.746ZINFOSledAgent (StorageMonitor): failed to notify nexus about a new pool oxp_31bd71cd-4736-4a12-a387-9b74b050396f on disk DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-31bd71cd-4736-4a12-a387-9b74b050396f", model: "synthetic-model" } err = "Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/sled-agents/e75545d5-2741-43ba-8c56-e7f2943a5b82/zpools/31bd71cd-4736-4a12-a387-9b74b050396f): error trying to connect: dns error: proto error: io error: No route to host (os error 148)" file = sled-agent/src/storage_monitor.rs:270
2132023-11-20T07:50:14.746ZINFOSledAgent (StorageMonitor): failed to notify nexus about a new pool oxp_cd70d7f6-2354-4bf2-8012-55bf9eaf7930 on disk DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-cd70d7f6-2354-4bf2-8012-55bf9eaf7930", model: "synthetic-model" } err = "Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/sled-agents/e75545d5-2741-43ba-8c56-e7f2943a5b82/zpools/cd70d7f6-2354-4bf2-8012-55bf9eaf7930): error trying to connect: dns error: proto error: io error: No route to host (os error 148)" file = sled-agent/src/storage_monitor.rs:270
2142023-11-20T07:50:14.746ZINFOSledAgent (StorageMonitor): failed to notify nexus about a new pool oxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03 on disk DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-f4b4dc87-ab46-49fb-a4b4-d361ae214c03", model: "synthetic-model" } err = "Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/sled-agents/e75545d5-2741-43ba-8c56-e7f2943a5b82/zpools/f4b4dc87-ab46-49fb-a4b4-d361ae214c03): error trying to connect: dns error: proto error: io error: No route to host (os error 148)" file = sled-agent/src/storage_monitor.rs:270
2152023-11-20T07:50:14.747ZINFOSledAgent (StorageMonitor): failed to notify nexus about Put(PhysicalDiskPutRequest { model: "synthetic-model", serial: "synthetic-serial-14b4dc87-ab46-49fb-a4b4-d361ae214c03", sled_id: e75545d5-2741-43ba-8c56-e7f2943a5b82, variant: U2, vendor: "synthetic-vendor" }) err = "Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/physical-disk): error trying to connect: dns error: proto error: io error: No route to host (os error 148)" file = sled-agent/src/storage_monitor.rs:219
2162023-11-20T07:50:14.747ZINFOSledAgent (StorageMonitor): failed to notify nexus about Put(PhysicalDiskPutRequest { model: "synthetic-model", serial: "synthetic-serial-24b4dc87-ab46-49fb-a4b4-d361ae214c03", sled_id: e75545d5-2741-43ba-8c56-e7f2943a5b82, variant: U2, vendor: "synthetic-vendor" }) err = "Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/physical-disk): error trying to connect: dns error: proto error: io error: No route to host (os error 148)" file = sled-agent/src/storage_monitor.rs:219
2172023-11-20T07:50:14.747ZINFOSledAgent (StorageMonitor): failed to notify nexus about Put(PhysicalDiskPutRequest { model: "synthetic-model", serial: "synthetic-serial-31bd71cd-4736-4a12-a387-9b74b050396f", sled_id: e75545d5-2741-43ba-8c56-e7f2943a5b82, variant: U2, vendor: "synthetic-vendor" }) err = "Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/physical-disk): error trying to connect: dns error: proto error: io error: No route to host (os error 148)" file = sled-agent/src/storage_monitor.rs:219
2182023-11-20T07:50:14.747ZINFOSledAgent (StorageMonitor): failed to notify nexus about Put(PhysicalDiskPutRequest { model: "synthetic-model", serial: "synthetic-serial-616b26df-e62a-4c68-b506-f4a923d8aaf7", sled_id: e75545d5-2741-43ba-8c56-e7f2943a5b82, variant: U2, vendor: "synthetic-vendor" }) err = "Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/physical-disk): error trying to connect: dns error: proto error: io error: No route to host (os error 148)" file = sled-agent/src/storage_monitor.rs:219
2192023-11-20T07:50:14.747ZINFOSledAgent (StorageMonitor): failed to notify nexus about Put(PhysicalDiskPutRequest { model: "synthetic-model", serial: "synthetic-serial-b462a7f7-b628-40fe-80ff-4e4189e2d62b", sled_id: e75545d5-2741-43ba-8c56-e7f2943a5b82, variant: M2, vendor: "synthetic-vendor" }) err = "Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/physical-disk): error trying to connect: dns error: proto error: io error: No route to host (os error 148)" file = sled-agent/src/storage_monitor.rs:219
2202023-11-20T07:50:14.747ZINFOSledAgent (StorageMonitor): failed to notify nexus about Put(PhysicalDiskPutRequest { model: "synthetic-model", serial: "synthetic-serial-cd70d7f6-2354-4bf2-8012-55bf9eaf7930", sled_id: e75545d5-2741-43ba-8c56-e7f2943a5b82, variant: U2, vendor: "synthetic-vendor" }) err = "Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/physical-disk): error trying to connect: dns error: proto error: io error: No route to host (os error 148)" file = sled-agent/src/storage_monitor.rs:219
2212023-11-20T07:50:14.747ZINFOSledAgent (StorageMonitor): failed to notify nexus about Put(PhysicalDiskPutRequest { model: "synthetic-model", serial: "synthetic-serial-ceb4461c-cf56-4719-ad3c-14430bfdfb60", sled_id: e75545d5-2741-43ba-8c56-e7f2943a5b82, variant: U2, vendor: "synthetic-vendor" }) err = "Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/physical-disk): error trying to connect: dns error: proto error: io error: No route to host (os error 148)" file = sled-agent/src/storage_monitor.rs:219
2222023-11-20T07:50:14.747ZINFOSledAgent (StorageMonitor): failed to notify nexus about Put(PhysicalDiskPutRequest { model: "synthetic-model", serial: "synthetic-serial-d462a7f7-b628-40fe-80ff-4e4189e2d62b", sled_id: e75545d5-2741-43ba-8c56-e7f2943a5b82, variant: U2, vendor: "synthetic-vendor" }) err = "Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/physical-disk): error trying to connect: dns error: proto error: io error: No route to host (os error 148)" file = sled-agent/src/storage_monitor.rs:219
2232023-11-20T07:50:14.747ZINFOSledAgent (StorageMonitor): failed to notify nexus about Put(PhysicalDiskPutRequest { model: "synthetic-model", serial: "synthetic-serial-e4b4dc87-ab46-49fb-a4b4-d361ae214c03", sled_id: e75545d5-2741-43ba-8c56-e7f2943a5b82, variant: U2, vendor: "synthetic-vendor" }) err = "Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/physical-disk): error trying to connect: dns error: proto error: io error: No route to host (os error 148)" file = sled-agent/src/storage_monitor.rs:219
2242023-11-20T07:50:14.748ZINFOSledAgent (StorageMonitor): failed to notify nexus about Put(PhysicalDiskPutRequest { model: "synthetic-model", serial: "synthetic-serial-f4b4dc87-ab46-49fb-a4b4-d361ae214c03", sled_id: e75545d5-2741-43ba-8c56-e7f2943a5b82, variant: U2, vendor: "synthetic-vendor" }) err = "Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/physical-disk): error trying to connect: dns error: proto error: io error: No route to host (os error 148)" file = sled-agent/src/storage_monitor.rs:219
2252023-11-20T07:50:14.748ZINFOSledAgent (StorageMonitor): failed to notify nexus about a new pool oxp_24b4dc87-ab46-49fb-a4b4-d361ae214c03 on disk DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-24b4dc87-ab46-49fb-a4b4-d361ae214c03", model: "synthetic-model" } err = "Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/sled-agents/e75545d5-2741-43ba-8c56-e7f2943a5b82/zpools/24b4dc87-ab46-49fb-a4b4-d361ae214c03): error trying to connect: dns error: proto error: io error: No route to host (os error 148)" file = sled-agent/src/storage_monitor.rs:270
2262023-11-20T07:50:14.748ZINFOSledAgent (StorageMonitor): failed to notify nexus about a new pool oxp_616b26df-e62a-4c68-b506-f4a923d8aaf7 on disk DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-616b26df-e62a-4c68-b506-f4a923d8aaf7", model: "synthetic-model" } err = "Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/sled-agents/e75545d5-2741-43ba-8c56-e7f2943a5b82/zpools/616b26df-e62a-4c68-b506-f4a923d8aaf7): error trying to connect: dns error: proto error: io error: No route to host (os error 148)" file = sled-agent/src/storage_monitor.rs:270
2272023-11-20T07:50:14.748ZINFOSledAgent (StorageMonitor): failed to notify nexus about a new pool oxi_a462a7f7-b628-40fe-80ff-4e4189e2d62b on disk DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-a462a7f7-b628-40fe-80ff-4e4189e2d62b", model: "synthetic-model" } err = "Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/sled-agents/e75545d5-2741-43ba-8c56-e7f2943a5b82/zpools/a462a7f7-b628-40fe-80ff-4e4189e2d62b): error trying to connect: dns error: proto error: io error: No route to host (os error 148)" file = sled-agent/src/storage_monitor.rs:270
2282023-11-20T07:50:14.748ZINFOSledAgent (StorageMonitor): failed to notify nexus about a new pool oxi_b462a7f7-b628-40fe-80ff-4e4189e2d62b on disk DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-b462a7f7-b628-40fe-80ff-4e4189e2d62b", model: "synthetic-model" } err = "Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/sled-agents/e75545d5-2741-43ba-8c56-e7f2943a5b82/zpools/b462a7f7-b628-40fe-80ff-4e4189e2d62b): error trying to connect: dns error: proto error: io error: No route to host (os error 148)" file = sled-agent/src/storage_monitor.rs:270
2292023-11-20T07:50:14.748ZINFOSledAgent (StorageMonitor): failed to notify nexus about a new pool oxp_ceb4461c-cf56-4719-ad3c-14430bfdfb60 on disk DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-ceb4461c-cf56-4719-ad3c-14430bfdfb60", model: "synthetic-model" } err = "Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/sled-agents/e75545d5-2741-43ba-8c56-e7f2943a5b82/zpools/ceb4461c-cf56-4719-ad3c-14430bfdfb60): error trying to connect: dns error: proto error: io error: No route to host (os error 148)" file = sled-agent/src/storage_monitor.rs:270
2302023-11-20T07:50:14.748ZINFOSledAgent (StorageMonitor): failed to notify nexus about a new pool oxp_d462a7f7-b628-40fe-80ff-4e4189e2d62b on disk DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-d462a7f7-b628-40fe-80ff-4e4189e2d62b", model: "synthetic-model" } err = "Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/sled-agents/e75545d5-2741-43ba-8c56-e7f2943a5b82/zpools/d462a7f7-b628-40fe-80ff-4e4189e2d62b): error trying to connect: dns error: proto error: io error: No route to host (os error 148)" file = sled-agent/src/storage_monitor.rs:270
2312023-11-20T07:50:14.748ZINFOSledAgent (StorageMonitor): failed to notify nexus about a new pool oxp_e4b4dc87-ab46-49fb-a4b4-d361ae214c03 on disk DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-e4b4dc87-ab46-49fb-a4b4-d361ae214c03", model: "synthetic-model" } err = "Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/sled-agents/e75545d5-2741-43ba-8c56-e7f2943a5b82/zpools/e4b4dc87-ab46-49fb-a4b4-d361ae214c03): error trying to connect: dns error: proto error: io error: No route to host (os error 148)" file = sled-agent/src/storage_monitor.rs:270
2322023-11-20T07:50:16.990ZINFOSledAgent (ServiceManager): Zone booting file = illumos-utils/src/running_zone.rs:509 zone = oxz_switch
2332023-11-20T07:50:17.385ZWARNSledAgent (ServiceManager): wait for service Some("oxz_switch") failed: Property not found. retry in 55.641995ms file = illumos-utils/src/svc.rs:36 zone = oxz_switch
2342023-11-20T07:50:17.386ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {}
2352023-11-20T07:50:17.596ZWARNSledAgent (ServiceManager): wait for service Some("oxz_switch") failed: Property not found. retry in 52.79937ms file = illumos-utils/src/svc.rs:36 zone = oxz_switch
2362023-11-20T07:50:17.786ZWARNSledAgent (ServiceManager): wait for service Some("oxz_switch") failed: Property not found. retry in 271.92687ms file = illumos-utils/src/svc.rs:36 zone = oxz_switch
2372023-11-20T07:50:18.174ZWARNSledAgent (ServiceManager): wait for service Some("oxz_switch") failed: Property not found. retry in 580.573175ms file = illumos-utils/src/svc.rs:36 zone = oxz_switch
2382023-11-20T07:50:18.820ZWARNSledAgent (ServiceManager): wait for service Some("oxz_switch") failed: Property not found. retry in 920.063447ms file = illumos-utils/src/svc.rs:36 zone = oxz_switch
2392023-11-20T07:50:19.800ZWARNSledAgent (ServiceManager): wait for service Some("oxz_switch") failed: Property not found. retry in 1.451420916s file = illumos-utils/src/svc.rs:36 zone = oxz_switch
2402023-11-20T07:50:21.282ZWARNSledAgent (ServiceManager): wait for service Some("oxz_switch") failed: Property not found. retry in 1.435012967s file = illumos-utils/src/svc.rs:36 zone = oxz_switch
2412023-11-20T07:50:22.387ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {}
2422023-11-20T07:50:23.702ZINFOSledAgent (ServiceManager): Ensuring tfportrear0_0/ll exists in zone file = sled-agent/src/services.rs:1324
2432023-11-20T07:50:25.016ZINFOSledAgent (ServiceManager): sled agent started file = sled-agent/src/services.rs:625 underlay_address = fd00:1122:3344:101::1
2442023-11-20T07:50:25.016ZINFOSledAgent (DnsResolver): new DNS resolver addresses = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53, [fd00:1122:3344:4::1]:53, [fd00:1122:3344:5::1]:53] file = internal-dns/src/resolver.rs:60
2452023-11-20T07:50:25.077ZINFOSledAgent (dropshot (SledAgent)): listening file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:195 local_addr = [fd00:1122:3344:101::1]:12345
2462023-11-20T07:50:25.077ZINFOSledAgent: contacting server nexus, registering sled baseboard = Baseboard { part_number: "i86pc", revision: 0, serial_number: "57495fb0328c443a91abff25dfea8602" } file = sled-agent/src/sled_agent.rs:626 id = e75545d5-2741-43ba-8c56-e7f2943a5b82 sled_id = e75545d5-2741-43ba-8c56-e7f2943a5b82
2472023-11-20T07:50:25.077ZINFOSledAgent (SledAgentMain): Sled Agent loaded; recording configuration file = sled-agent/src/bootstrap/server.rs:424
2482023-11-20T07:50:25.077ZINFOSledAgent (StorageManager): Received GetLatestResources(Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) }) file = sled-storage/src/manager.rs:346
2492023-11-20T07:50:25.077ZINFOSledAgent (SledAgentMain): Writing ledger to /pool/int/a462a7f7-b628-40fe-80ff-4e4189e2d62b/config/.sled-agent-request.json.tmp file = common/src/ledger.rs:197
2502023-11-20T07:50:25.077ZWARNSledAgent: failed to register as a metric producer with Nexus error = "Metric registration error: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/metrics/producers): error trying to connect: dns error: proto error: io error: No route to host (os error 148)" file = sled-agent/src/sled_agent.rs:1053 retry_after = 330.466845ms sled_id = e75545d5-2741-43ba-8c56-e7f2943a5b82
2512023-11-20T07:50:25.078ZINFOSledAgent (SledAgentMain): Writing ledger to /pool/int/b462a7f7-b628-40fe-80ff-4e4189e2d62b/config/.sled-agent-request.json.tmp file = common/src/ledger.rs:197
2522023-11-20T07:50:25.078ZINFOSledAgent (HardwareMonitor): Sled Agent Started file = sled-agent/src/hardware_monitor.rs:137
2532023-11-20T07:50:25.078ZINFOSledAgent (HardwareMonitor): Checking current full hardware snapshot file = sled-agent/src/hardware_monitor.rs:241 underlay_network_info = Some((fd00:1122:3344:101::2, Some(RackNetworkConfigV1 { rack_subnet: Ipv6Network { addr: fd00:1122:3344:1::, prefix: 56 }, infra_ip_first: 10.151.2.174, infra_ip_last: 10.151.2.174, ports: [PortConfigV1 { routes: [RouteConfig { destination: V4(Ipv4Network { addr: 0.0.0.0, prefix: 0 }), nexthop: 10.151.2.100 }], addresses: [V4(Ipv4Network { addr: 10.151.2.174, prefix: 32 })], switch: Switch0, port: "qsfp0", uplink_port_speed: Speed40G, uplink_port_fec: None, bgp_peers: [] }], bgp: [] })))
2542023-11-20T07:50:25.078ZINFOSledAgent (ServiceManager): Ensuring scrimlet services (enabling services) file = sled-agent/src/services.rs:2520
2552023-11-20T07:50:25.078ZINFOSledAgent (SledAgentSprocketsServer): Connection closed file = sled-agent/src/bootstrap/sprockets_server.rs:73 remote_addr = [fdb0:18c0:4d0d:a02a::1]:46844
2562023-11-20T07:50:25.081ZINFOSledAgent (BootstrapAgentRssHandler): Peer agent initialized file = sled-agent/src/bootstrap/rss_handle.rs:108 peer = [fdb0:18c0:4d0d:a02a::1]:12346
2572023-11-20T07:50:25.081ZINFOSledAgent (BootstrapAgentRssHandler): Initialized sled agent file = sled-agent/src/bootstrap/rss_handle.rs:232 target_sled = [fdb0:18c0:4d0d:a02a::1]:12346
2582023-11-20T07:50:25.081ZINFOSledAgent (StorageManager): Received GetLatestResources(Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) }) file = sled-storage/src/manager.rs:346
2592023-11-20T07:50:25.081ZWARNSledAgent (RSS): No ledger in /pool/int/a462a7f7-b628-40fe-80ff-4e4189e2d62b/config/rss-service-plan.json file = common/src/ledger.rs:186
2602023-11-20T07:50:25.081ZERROSledAgent (RSS): Failed to read ledger: Not found in storage file = common/src/ledger.rs:90 path = /pool/int/a462a7f7-b628-40fe-80ff-4e4189e2d62b/config/rss-service-plan.json
2612023-11-20T07:50:25.081ZWARNSledAgent (RSS): No ledger in /pool/int/b462a7f7-b628-40fe-80ff-4e4189e2d62b/config/rss-service-plan.json file = common/src/ledger.rs:186
2622023-11-20T07:50:25.081ZERROSledAgent (RSS): Failed to read ledger: Not found in storage file = common/src/ledger.rs:90 path = /pool/int/b462a7f7-b628-40fe-80ff-4e4189e2d62b/config/rss-service-plan.json
2632023-11-20T07:50:25.082ZINFOSledAgent: failed to notify nexus about sled agent error = Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/sled-agents/e75545d5-2741-43ba-8c56-e7f2943a5b82): error trying to connect: dns error: proto error: io error: No route to host (os error 148) file = sled-agent/src/sled_agent.rs:663 sled_id = e75545d5-2741-43ba-8c56-e7f2943a5b82
2642023-11-20T07:50:25.114ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:769 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:59287
2652023-11-20T07:50:25.115ZINFOSledAgent (StorageManager): Received GetLatestResources(Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) }) file = sled-storage/src/manager.rs:346
2662023-11-20T07:50:25.115ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 147 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:59287 req_id = 4520455e-a018-4b98-acbc-20477258e197 response_code = 200 uri = /zpools
2672023-11-20T07:50:25.144ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:769 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:57005
2682023-11-20T07:50:25.145ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 472 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:57005 req_id = b929c7ee-0d91-47cd-9504-ca4b5b7181e9 response_code = 200 uri = /sled-role
2692023-11-20T07:50:25.147ZINFOSledAgent (StorageManager): Received GetLatestResources(Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) }) file = sled-storage/src/manager.rs:346
2702023-11-20T07:50:25.147ZINFOSledAgent (RSS): Writing ledger to /pool/int/a462a7f7-b628-40fe-80ff-4e4189e2d62b/config/.rss-service-plan.json.tmp file = common/src/ledger.rs:197
2712023-11-20T07:50:25.148ZINFOSledAgent (RSS): Writing ledger to /pool/int/b462a7f7-b628-40fe-80ff-4e4189e2d62b/config/.rss-service-plan.json.tmp file = common/src/ledger.rs:197
2722023-11-20T07:50:25.148ZINFOSledAgent (RSS): Service plan written to storage file = sled-agent/src/rack_setup/plan/service.rs:750
2732023-11-20T07:50:25.179ZINFOSledAgent (RSS): sending service requests... file = sled-agent/src/rack_setup/service.rs:284
2742023-11-20T07:50:25.179ZINFOSledAgent (RSS): initializing sled services: [ServiceZoneRequest { addresses: [fd00:1122:3344:1::1], dataset: Some(DatasetRequest { id: c9a9027a-c2d6-437c-b4f4-de4f53331cd0, name: DatasetName { kind: InternalDns, pool_name: ZpoolName("oxp_14b4dc87-ab46-49fb-a4b4-d361ae214c03") }, service_address: "[fd00:1122:3344:1::1]:5353" }), id: c9a9027a-c2d6-437c-b4f4-de4f53331cd0, services: [ServiceZoneService { details: InternalDns { dns_address: "[fd00:1122:3344:1::1]:53", gz_address: fd00:1122:3344:1::2, gz_address_index: 0, http_address: "[fd00:1122:3344:1::1]:5353" }, id: c9a9027a-c2d6-437c-b4f4-de4f53331cd0 }], zone_type: InternalDns }, ServiceZoneRequest { addresses: [fd00:1122:3344:2::1], dataset: Some(DatasetRequest { id: defece7e-1f41-48b6-a56f-7f3b68d5a311, name: DatasetName { kind: InternalDns, pool_name: ZpoolName("oxp_24b4dc87-ab46-49fb-a4b4-d361ae214c03") }, service_address: "[fd00:1122:3344:2::1]:5353" }), id: defece7e-1f41-48b6-a56f-7f3b68d5a311, services: [ServiceZoneService { details: InternalDns { dns_address: "[fd00:1122:3344:2::1]:53", gz_address: fd00:1122:3344:2::2, gz_address_index: 1, http_address: "[fd00:1122:3344:2::1]:5353" }, id: defece7e-1f41-48b6-a56f-7f3b68d5a311 }], zone_type: InternalDns }, ServiceZoneRequest { addresses: [fd00:1122:3344:3::1], dataset: Some(DatasetRequest { id: 306c7298-6fe3-4323-b3ea-7a7f9075ced3, name: DatasetName { kind: InternalDns, pool_name: ZpoolName("oxp_31bd71cd-4736-4a12-a387-9b74b050396f") }, service_address: "[fd00:1122:3344:3::1]:5353" }), id: 306c7298-6fe3-4323-b3ea-7a7f9075ced3, services: [ServiceZoneService { details: InternalDns { dns_address: "[fd00:1122:3344:3::1]:53", gz_address: fd00:1122:3344:3::2, gz_address_index: 2, http_address: "[fd00:1122:3344:3::1]:5353" }, id: 306c7298-6fe3-4323-b3ea-7a7f9075ced3 }], zone_type: InternalDns }] file = sled-agent/src/rack_setup/service.rs:286
2752023-11-20T07:50:25.179ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:769 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:62523
2762023-11-20T07:50:25.179ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: c9a9027a-c2d6-437c-b4f4-de4f53331cd0, dataset_name: DatasetName { pool_name: ZpoolName { id: 14b4dc87-ab46-49fb-a4b4-d361ae214c03, kind: External }, kind: InternalDns }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } }) file = sled-storage/src/manager.rs:346
2772023-11-20T07:50:25.179ZINFOSledAgent (StorageManager): add_dataset: NewFilesystemRequest { dataset_id: c9a9027a-c2d6-437c-b4f4-de4f53331cd0, dataset_name: DatasetName { pool_name: ZpoolName { id: 14b4dc87-ab46-49fb-a4b4-d361ae214c03, kind: External }, kind: InternalDns }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } } file = sled-storage/src/manager.rs:564
2782023-11-20T07:50:25.231ZINFOSledAgent: contacting server nexus, registering sled baseboard = Baseboard { part_number: "i86pc", revision: 0, serial_number: "57495fb0328c443a91abff25dfea8602" } file = sled-agent/src/sled_agent.rs:626 id = e75545d5-2741-43ba-8c56-e7f2943a5b82 sled_id = e75545d5-2741-43ba-8c56-e7f2943a5b82
2792023-11-20T07:50:25.231ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: defece7e-1f41-48b6-a56f-7f3b68d5a311, dataset_name: DatasetName { pool_name: ZpoolName { id: 24b4dc87-ab46-49fb-a4b4-d361ae214c03, kind: External }, kind: InternalDns }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } }) file = sled-storage/src/manager.rs:346
2802023-11-20T07:50:25.231ZINFOSledAgent (StorageManager): add_dataset: NewFilesystemRequest { dataset_id: defece7e-1f41-48b6-a56f-7f3b68d5a311, dataset_name: DatasetName { pool_name: ZpoolName { id: 24b4dc87-ab46-49fb-a4b4-d361ae214c03, kind: External }, kind: InternalDns }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } } file = sled-storage/src/manager.rs:564
2812023-11-20T07:50:25.279ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 306c7298-6fe3-4323-b3ea-7a7f9075ced3, dataset_name: DatasetName { pool_name: ZpoolName { id: 31bd71cd-4736-4a12-a387-9b74b050396f, kind: External }, kind: InternalDns }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } }) file = sled-storage/src/manager.rs:346
2822023-11-20T07:50:25.279ZINFOSledAgent (StorageManager): add_dataset: NewFilesystemRequest { dataset_id: 306c7298-6fe3-4323-b3ea-7a7f9075ced3, dataset_name: DatasetName { pool_name: ZpoolName { id: 31bd71cd-4736-4a12-a387-9b74b050396f, kind: External }, kind: InternalDns }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } } file = sled-storage/src/manager.rs:564
2832023-11-20T07:50:25.326ZINFOSledAgent (StorageManager): Received GetLatestResources(Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) }) file = sled-storage/src/manager.rs:346
2842023-11-20T07:50:25.326ZWARNSledAgent (ServiceManager): No ledger in /pool/int/a462a7f7-b628-40fe-80ff-4e4189e2d62b/config/services.json file = common/src/ledger.rs:186
2852023-11-20T07:50:25.326ZERROSledAgent (ServiceManager): Failed to read ledger: Not found in storage file = common/src/ledger.rs:90 path = /pool/int/a462a7f7-b628-40fe-80ff-4e4189e2d62b/config/services.json
2862023-11-20T07:50:25.326ZWARNSledAgent (ServiceManager): No ledger in /pool/int/b462a7f7-b628-40fe-80ff-4e4189e2d62b/config/services.json file = common/src/ledger.rs:186
2872023-11-20T07:50:25.326ZERROSledAgent (ServiceManager): Failed to read ledger: Not found in storage file = common/src/ledger.rs:90 path = /pool/int/b462a7f7-b628-40fe-80ff-4e4189e2d62b/config/services.json
2882023-11-20T07:50:25.326ZINFOSledAgent (StorageManager): Received GetLatestResources(Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) }) file = sled-storage/src/manager.rs:346
2892023-11-20T07:50:25.331ZINFOSledAgent (StorageManager): Received GetLatestResources(Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) }) file = sled-storage/src/manager.rs:346
2902023-11-20T07:50:25.331ZINFOSledAgent (StorageManager): Received GetLatestResources(Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) }) file = sled-storage/src/manager.rs:346
2912023-11-20T07:50:25.331ZINFOSledAgent (StorageManager): Received GetLatestResources(Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) }) file = sled-storage/src/manager.rs:346
2922023-11-20T07:50:25.334ZINFOSledAgent: contacting server nexus, registering sled baseboard = Baseboard { part_number: "i86pc", revision: 0, serial_number: "57495fb0328c443a91abff25dfea8602" } file = sled-agent/src/sled_agent.rs:626 id = e75545d5-2741-43ba-8c56-e7f2943a5b82 sled_id = e75545d5-2741-43ba-8c56-e7f2943a5b82
2932023-11-20T07:50:25.386ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_internal_dns_c9a9027a-c2d6-437c-b4f4-de4f53331cd0 file = illumos-utils/src/zone.rs:314
2942023-11-20T07:50:25.390ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_internal_dns_defece7e-1f41-48b6-a56f-7f3b68d5a311 file = illumos-utils/src/zone.rs:314
2952023-11-20T07:50:25.393ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_internal_dns_306c7298-6fe3-4323-b3ea-7a7f9075ced3 file = illumos-utils/src/zone.rs:314
2962023-11-20T07:50:25.405ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_internal_dns_c9a9027a-c2d6-437c-b4f4-de4f53331cd0 file = illumos-utils/src/zone.rs:353
2972023-11-20T07:50:25.410ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_internal_dns_defece7e-1f41-48b6-a56f-7f3b68d5a311 file = illumos-utils/src/zone.rs:353
2982023-11-20T07:50:25.414ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_internal_dns_306c7298-6fe3-4323-b3ea-7a7f9075ced3 file = illumos-utils/src/zone.rs:353
2992023-11-20T07:50:25.417ZWARNSledAgent: failed to register as a metric producer with Nexus error = "Metric registration error: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/metrics/producers): error trying to connect: dns error: proto error: io error: No route to host (os error 148)" file = sled-agent/src/sled_agent.rs:1053 retry_after = 405.758773ms sled_id = e75545d5-2741-43ba-8c56-e7f2943a5b82
3002023-11-20T07:50:25.418ZINFOSledAgent: contacting server nexus, registering sled baseboard = Baseboard { part_number: "i86pc", revision: 0, serial_number: "57495fb0328c443a91abff25dfea8602" } file = sled-agent/src/sled_agent.rs:626 id = e75545d5-2741-43ba-8c56-e7f2943a5b82 sled_id = e75545d5-2741-43ba-8c56-e7f2943a5b82
3012023-11-20T07:50:25.674ZINFOSledAgent: contacting server nexus, registering sled baseboard = Baseboard { part_number: "i86pc", revision: 0, serial_number: "57495fb0328c443a91abff25dfea8602" } file = sled-agent/src/sled_agent.rs:626 id = e75545d5-2741-43ba-8c56-e7f2943a5b82 sled_id = e75545d5-2741-43ba-8c56-e7f2943a5b82
3022023-11-20T07:50:25.825ZWARNSledAgent: failed to register as a metric producer with Nexus error = "Metric registration error: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/metrics/producers): error trying to connect: dns error: proto error: io error: No route to host (os error 148)" file = sled-agent/src/sled_agent.rs:1053 retry_after = 1.282920044s sled_id = e75545d5-2741-43ba-8c56-e7f2943a5b82
3032023-11-20T07:50:25.966ZINFOSledAgent: contacting server nexus, registering sled baseboard = Baseboard { part_number: "i86pc", revision: 0, serial_number: "57495fb0328c443a91abff25dfea8602" } file = sled-agent/src/sled_agent.rs:626 id = e75545d5-2741-43ba-8c56-e7f2943a5b82 sled_id = e75545d5-2741-43ba-8c56-e7f2943a5b82
3042023-11-20T07:50:26.256ZINFOSledAgent: contacting server nexus, registering sled baseboard = Baseboard { part_number: "i86pc", revision: 0, serial_number: "57495fb0328c443a91abff25dfea8602" } file = sled-agent/src/sled_agent.rs:626 id = e75545d5-2741-43ba-8c56-e7f2943a5b82 sled_id = e75545d5-2741-43ba-8c56-e7f2943a5b82
3052023-11-20T07:50:26.703ZINFOSledAgent: contacting server nexus, registering sled baseboard = Baseboard { part_number: "i86pc", revision: 0, serial_number: "57495fb0328c443a91abff25dfea8602" } file = sled-agent/src/sled_agent.rs:626 id = e75545d5-2741-43ba-8c56-e7f2943a5b82 sled_id = e75545d5-2741-43ba-8c56-e7f2943a5b82
3062023-11-20T07:50:27.060ZINFOSledAgent (ServiceManager): Zone booting file = illumos-utils/src/running_zone.rs:509 zone = oxz_internal_dns_c9a9027a-c2d6-437c-b4f4-de4f53331cd0
3072023-11-20T07:50:27.078ZINFOSledAgent: contacting server nexus, registering sled baseboard = Baseboard { part_number: "i86pc", revision: 0, serial_number: "57495fb0328c443a91abff25dfea8602" } file = sled-agent/src/sled_agent.rs:626 id = e75545d5-2741-43ba-8c56-e7f2943a5b82 sled_id = e75545d5-2741-43ba-8c56-e7f2943a5b82
3082023-11-20T07:50:27.110ZWARNSledAgent: failed to register as a metric producer with Nexus error = "Metric registration error: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/metrics/producers): error trying to connect: dns error: proto error: io error: No route to host (os error 148)" file = sled-agent/src/sled_agent.rs:1053 retry_after = 2.65674888s sled_id = e75545d5-2741-43ba-8c56-e7f2943a5b82
3092023-11-20T07:50:27.173ZINFOSledAgent (ServiceManager): Zone booting file = illumos-utils/src/running_zone.rs:509 zone = oxz_internal_dns_306c7298-6fe3-4323-b3ea-7a7f9075ced3
3102023-11-20T07:50:27.179ZINFOSledAgent (ServiceManager): Zone booting file = illumos-utils/src/running_zone.rs:509 zone = oxz_internal_dns_defece7e-1f41-48b6-a56f-7f3b68d5a311
3112023-11-20T07:50:27.199ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_c9a9027a-c2d6-437c-b4f4-de4f53331cd0") failed: Property not found. retry in 57.154693ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_c9a9027a-c2d6-437c-b4f4-de4f53331cd0
3122023-11-20T07:50:27.267ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_c9a9027a-c2d6-437c-b4f4-de4f53331cd0") failed: Property not found. retry in 75.074371ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_c9a9027a-c2d6-437c-b4f4-de4f53331cd0
3132023-11-20T07:50:27.392ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_c9a9027a-c2d6-437c-b4f4-de4f53331cd0") failed: Property not found. retry in 242.442145ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_c9a9027a-c2d6-437c-b4f4-de4f53331cd0
3142023-11-20T07:50:27.392ZINFOSledAgent: contacting server nexus, registering sled baseboard = Baseboard { part_number: "i86pc", revision: 0, serial_number: "57495fb0328c443a91abff25dfea8602" } file = sled-agent/src/sled_agent.rs:626 id = e75545d5-2741-43ba-8c56-e7f2943a5b82 sled_id = e75545d5-2741-43ba-8c56-e7f2943a5b82
3152023-11-20T07:50:27.397ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {}
3162023-11-20T07:50:27.403ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_defece7e-1f41-48b6-a56f-7f3b68d5a311") failed: Property not found. retry in 32.926668ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_defece7e-1f41-48b6-a56f-7f3b68d5a311
3172023-11-20T07:50:27.412ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_306c7298-6fe3-4323-b3ea-7a7f9075ced3") failed: Property not found. retry in 73.88137ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_306c7298-6fe3-4323-b3ea-7a7f9075ced3
3182023-11-20T07:50:27.470ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_defece7e-1f41-48b6-a56f-7f3b68d5a311") failed: Property not found. retry in 54.166435ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_defece7e-1f41-48b6-a56f-7f3b68d5a311
3192023-11-20T07:50:27.518ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_306c7298-6fe3-4323-b3ea-7a7f9075ced3") failed: Property not found. retry in 87.941042ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_306c7298-6fe3-4323-b3ea-7a7f9075ced3
3202023-11-20T07:50:27.562ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_defece7e-1f41-48b6-a56f-7f3b68d5a311") failed: Property not found. retry in 133.674121ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_defece7e-1f41-48b6-a56f-7f3b68d5a311
3212023-11-20T07:50:27.654ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_306c7298-6fe3-4323-b3ea-7a7f9075ced3") failed: Property not found. retry in 284.099185ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_306c7298-6fe3-4323-b3ea-7a7f9075ced3
3222023-11-20T07:50:27.690ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_c9a9027a-c2d6-437c-b4f4-de4f53331cd0") failed: Property not found. retry in 394.471652ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_c9a9027a-c2d6-437c-b4f4-de4f53331cd0
3232023-11-20T07:50:27.728ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_defece7e-1f41-48b6-a56f-7f3b68d5a311") failed: Property not found. retry in 400.394545ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_defece7e-1f41-48b6-a56f-7f3b68d5a311
3242023-11-20T07:50:27.947ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_306c7298-6fe3-4323-b3ea-7a7f9075ced3") failed: Property not found. retry in 270.817983ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_306c7298-6fe3-4323-b3ea-7a7f9075ced3
3252023-11-20T07:50:28.094ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_c9a9027a-c2d6-437c-b4f4-de4f53331cd0") failed: Property not found. retry in 809.534101ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_c9a9027a-c2d6-437c-b4f4-de4f53331cd0
3262023-11-20T07:50:28.136ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_defece7e-1f41-48b6-a56f-7f3b68d5a311") failed: Property not found. retry in 629.5388ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_defece7e-1f41-48b6-a56f-7f3b68d5a311
3272023-11-20T07:50:28.160ZINFOSledAgent: contacting server nexus, registering sled baseboard = Baseboard { part_number: "i86pc", revision: 0, serial_number: "57495fb0328c443a91abff25dfea8602" } file = sled-agent/src/sled_agent.rs:626 id = e75545d5-2741-43ba-8c56-e7f2943a5b82 sled_id = e75545d5-2741-43ba-8c56-e7f2943a5b82
3282023-11-20T07:50:28.227ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_306c7298-6fe3-4323-b3ea-7a7f9075ced3") failed: Property not found. retry in 506.632217ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_306c7298-6fe3-4323-b3ea-7a7f9075ced3
3292023-11-20T07:50:28.399ZINFOSledAgent (ServiceManager): Ensuring bootstrap address fdb0:18c0:4d0d:a02a::2 exists in switch zone file = sled-agent/src/services.rs:1341
3302023-11-20T07:50:28.399ZINFOSledAgent (ServiceManager): Adding bootstrap address file = illumos-utils/src/running_zone.rs:622 zone = oxz_switch
3312023-11-20T07:50:28.533ZINFOSledAgent: contacting server nexus, registering sled baseboard = Baseboard { part_number: "i86pc", revision: 0, serial_number: "57495fb0328c443a91abff25dfea8602" } file = sled-agent/src/sled_agent.rs:626 id = e75545d5-2741-43ba-8c56-e7f2943a5b82 sled_id = e75545d5-2741-43ba-8c56-e7f2943a5b82
3322023-11-20T07:50:28.745ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_306c7298-6fe3-4323-b3ea-7a7f9075ced3") failed: Property not found. retry in 1.467585721s file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_306c7298-6fe3-4323-b3ea-7a7f9075ced3
3332023-11-20T07:50:28.777ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_defece7e-1f41-48b6-a56f-7f3b68d5a311") failed: Property not found. retry in 576.516395ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_defece7e-1f41-48b6-a56f-7f3b68d5a311
3342023-11-20T07:50:28.914ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_c9a9027a-c2d6-437c-b4f4-de4f53331cd0") failed: Property not found. retry in 848.475116ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_c9a9027a-c2d6-437c-b4f4-de4f53331cd0
3352023-11-20T07:50:28.971ZINFOSledAgent: contacting server nexus, registering sled baseboard = Baseboard { part_number: "i86pc", revision: 0, serial_number: "57495fb0328c443a91abff25dfea8602" } file = sled-agent/src/sled_agent.rs:626 id = e75545d5-2741-43ba-8c56-e7f2943a5b82 sled_id = e75545d5-2741-43ba-8c56-e7f2943a5b82
3362023-11-20T07:50:29.363ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_defece7e-1f41-48b6-a56f-7f3b68d5a311") failed: Property not found. retry in 742.105665ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_defece7e-1f41-48b6-a56f-7f3b68d5a311
3372023-11-20T07:50:29.772ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_c9a9027a-c2d6-437c-b4f4-de4f53331cd0") failed: Property not found. retry in 1.471130483s file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_c9a9027a-c2d6-437c-b4f4-de4f53331cd0
3382023-11-20T07:50:29.774ZWARNSledAgent: failed to register as a metric producer with Nexus error = "Metric registration error: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/metrics/producers): error trying to connect: dns error: proto error: io error: No route to host (os error 148)" file = sled-agent/src/sled_agent.rs:1053 retry_after = 4.488084432s sled_id = e75545d5-2741-43ba-8c56-e7f2943a5b82
3392023-11-20T07:50:30.113ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_defece7e-1f41-48b6-a56f-7f3b68d5a311") failed: Property not found. retry in 1.041011768s file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_defece7e-1f41-48b6-a56f-7f3b68d5a311
3402023-11-20T07:50:30.219ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_306c7298-6fe3-4323-b3ea-7a7f9075ced3") failed: Property not found. retry in 1.009341734s file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_306c7298-6fe3-4323-b3ea-7a7f9075ced3
3412023-11-20T07:50:30.285ZINFOSledAgent: contacting server nexus, registering sled baseboard = Baseboard { part_number: "i86pc", revision: 0, serial_number: "57495fb0328c443a91abff25dfea8602" } file = sled-agent/src/sled_agent.rs:626 id = e75545d5-2741-43ba-8c56-e7f2943a5b82 sled_id = e75545d5-2741-43ba-8c56-e7f2943a5b82
3422023-11-20T07:50:31.044ZINFOSledAgent: contacting server nexus, registering sled baseboard = Baseboard { part_number: "i86pc", revision: 0, serial_number: "57495fb0328c443a91abff25dfea8602" } file = sled-agent/src/sled_agent.rs:626 id = e75545d5-2741-43ba-8c56-e7f2943a5b82 sled_id = e75545d5-2741-43ba-8c56-e7f2943a5b82
3432023-11-20T07:50:31.165ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_defece7e-1f41-48b6-a56f-7f3b68d5a311") failed: Property not found. retry in 876.457772ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_defece7e-1f41-48b6-a56f-7f3b68d5a311
3442023-11-20T07:50:31.238ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_306c7298-6fe3-4323-b3ea-7a7f9075ced3") failed: Property not found. retry in 764.319614ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_306c7298-6fe3-4323-b3ea-7a7f9075ced3
3452023-11-20T07:50:31.252ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_c9a9027a-c2d6-437c-b4f4-de4f53331cd0") failed: Property not found. retry in 736.243111ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_c9a9027a-c2d6-437c-b4f4-de4f53331cd0
3462023-11-20T07:50:32.049ZINFOSledAgent (ServiceManager): Ensuring address fd00:1122:3344:1::1 exists file = sled-agent/src/services.rs:1370
3472023-11-20T07:50:32.049ZINFOSledAgent (ServiceManager): Adding address: Static(V6(Ipv6Network { addr: fd00:1122:3344:1::1, prefix: 64 })) file = illumos-utils/src/running_zone.rs:605 zone = oxz_internal_dns_c9a9027a-c2d6-437c-b4f4-de4f53331cd0
3482023-11-20T07:50:32.612ZINFOSledAgent (ServiceManager): Ensuring address fd00:1122:3344:1::1 exists - OK file = sled-agent/src/services.rs:1378
3492023-11-20T07:50:32.612ZINFOSledAgent (ServiceManager): Zone not using gateway (even though underlay is up) file = sled-agent/src/services.rs:1404
3502023-11-20T07:50:32.786ZINFOSledAgent (ServiceManager): Setting up internal-dns service file = sled-agent/src/services.rs:1577
3512023-11-20T07:50:32.931ZINFOSledAgent (ServiceManager): Forwarding bootstrap traffic via oxBootstrap0 to fe80::8:20ff:fe0b:905c file = sled-agent/src/services.rs:1348
3522023-11-20T07:50:32.938ZINFOSledAgent (ServiceManager): Zone not using gateway (even though underlay is up) file = sled-agent/src/services.rs:1404
3532023-11-20T07:50:33.058ZINFOSledAgent (ServiceManager): Setting up dendrite service file = sled-agent/src/services.rs:1757
3542023-11-20T07:50:33.305ZINFOSledAgent (ServiceManager): Setting up MGS service file = sled-agent/src/services.rs:1645
3552023-11-20T07:50:33.333ZINFOSledAgent (BootstrapAgentStartup): Sending prefix to ddmd for advertisement DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:91 prefix = Ipv6Prefix { addr: fd00:1122:3344:1::, len: 64 }
3562023-11-20T07:50:33.333ZINFOSledAgent: contacting server nexus, registering sled baseboard = Baseboard { part_number: "i86pc", revision: 0, serial_number: "57495fb0328c443a91abff25dfea8602" } file = sled-agent/src/sled_agent.rs:626 id = e75545d5-2741-43ba-8c56-e7f2943a5b82 sled_id = e75545d5-2741-43ba-8c56-e7f2943a5b82
3572023-11-20T07:50:33.338ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {}
3582023-11-20T07:50:33.372ZINFOSledAgent (ServiceManager): Ensuring address fd00:1122:3344:3::1 exists file = sled-agent/src/services.rs:1370
3592023-11-20T07:50:33.372ZINFOSledAgent (ServiceManager): Adding address: Static(V6(Ipv6Network { addr: fd00:1122:3344:3::1, prefix: 64 })) file = illumos-utils/src/running_zone.rs:605 zone = oxz_internal_dns_306c7298-6fe3-4323-b3ea-7a7f9075ced3
3602023-11-20T07:50:33.602ZINFOSledAgent (ServiceManager): Setting up wicketd service file = sled-agent/src/services.rs:1675
3612023-11-20T07:50:33.791ZINFOSledAgent (ServiceManager): Setting up mgd service file = sled-agent/src/services.rs:1981
3622023-11-20T07:50:33.917ZINFOSledAgent (ServiceManager): Ensuring address fd00:1122:3344:3::1 exists - OK file = sled-agent/src/services.rs:1378
3632023-11-20T07:50:33.917ZINFOSledAgent (ServiceManager): Zone not using gateway (even though underlay is up) file = sled-agent/src/services.rs:1404
3642023-11-20T07:50:33.935ZINFOSledAgent (ServiceManager): Setting up mg-ddm service file = sled-agent/src/services.rs:1986
3652023-11-20T07:50:34.092ZINFOSledAgent (ServiceManager): Setting up internal-dns service file = sled-agent/src/services.rs:1577
3662023-11-20T07:50:34.113ZINFOSledAgent (ServiceManager): Setting up tfport service file = sled-agent/src/services.rs:1867
3672023-11-20T07:50:34.281ZINFOSledAgent (ServiceManager): Setting up Simulated SP service file = sled-agent/src/services.rs:1672
3682023-11-20T07:50:34.291ZINFOSledAgent (ServiceManager): Re-enabling running switch zone (new address) file = sled-agent/src/services.rs:2789 new = [fd00:1122:3344:101::2, ::1] old = [::1]
3692023-11-20T07:50:34.291ZINFOSledAgent (ServiceManager): Ensuring address fd00:1122:3344:101::2 exists file = sled-agent/src/services.rs:2805
3702023-11-20T07:50:34.291ZINFOSledAgent (ServiceManager): Adding address: Static(V6(Ipv6Network { addr: fd00:1122:3344:101::2, prefix: 64 })) file = illumos-utils/src/running_zone.rs:605 zone = oxz_switch
3712023-11-20T07:50:34.621ZINFOSledAgent (BootstrapAgentStartup): Sending prefix to ddmd for advertisement DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:91 prefix = Ipv6Prefix { addr: fd00:1122:3344:3::, len: 64 }
3722023-11-20T07:50:34.645ZINFOSledAgent (ServiceManager): Ensuring address fd00:1122:3344:2::1 exists file = sled-agent/src/services.rs:1370
3732023-11-20T07:50:34.645ZINFOSledAgent (ServiceManager): Adding address: Static(V6(Ipv6Network { addr: fd00:1122:3344:2::1, prefix: 64 })) file = illumos-utils/src/running_zone.rs:605 zone = oxz_internal_dns_defece7e-1f41-48b6-a56f-7f3b68d5a311
3742023-11-20T07:50:34.836ZINFOSledAgent (ServiceManager): Ensuring address fd00:1122:3344:101::2 exists - OK file = sled-agent/src/services.rs:2813
3752023-11-20T07:50:34.842ZINFOSledAgent (ServiceManager): configuring dendrite zone file = sled-agent/src/services.rs:2868
3762023-11-20T07:50:35.025ZINFOSledAgent (ServiceManager): configuring wicketd file = sled-agent/src/services.rs:2908 rack_subnet = fd00:1122:3344::
3772023-11-20T07:50:35.078ZINFOSledAgent (ServiceManager): Determining physical location of our switch zone at fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:359
3782023-11-20T07:50:35.112ZWARNSledAgent (ServiceManager): Failed to get switch ID from MGS (retrying in 57.370631ms) error = Error Response: status: 503 Service Unavailable; headers: {"content-type": "application/json", "x-request-id": "04f8ee71-e9d2-4f7e-961d-e58c9f10eb43", "content-length": "152", "date": "Mon, 20 Nov 2023 07:50:34 GMT"}; value: Error { error_code: Some("DiscoveryNotYetComplete"), message: "discovery process not yet complete", request_id: "04f8ee71-e9d2-4f7e-961d-e58c9f10eb43" } file = sled-agent/src/bootstrap/early_networking.rs:378
3792023-11-20T07:50:35.171ZWARNSledAgent (ServiceManager): Failed to get switch ID from MGS (retrying in 94.566651ms) error = Error Response: status: 503 Service Unavailable; headers: {"content-type": "application/json", "x-request-id": "8a6ad58c-a5e3-415d-b836-ee7905bfda1e", "content-length": "152", "date": "Mon, 20 Nov 2023 07:50:34 GMT"}; value: Error { error_code: Some("DiscoveryNotYetComplete"), message: "discovery process not yet complete", request_id: "8a6ad58c-a5e3-415d-b836-ee7905bfda1e" } file = sled-agent/src/bootstrap/early_networking.rs:378
3802023-11-20T07:50:35.193ZINFOSledAgent (ServiceManager): Ensuring address fd00:1122:3344:2::1 exists - OK file = sled-agent/src/services.rs:1378
3812023-11-20T07:50:35.193ZINFOSledAgent (ServiceManager): Zone not using gateway (even though underlay is up) file = sled-agent/src/services.rs:1404
3822023-11-20T07:50:35.268ZWARNSledAgent (ServiceManager): Failed to get switch ID from MGS (retrying in 289.745893ms) error = Error Response: status: 503 Service Unavailable; headers: {"content-type": "application/json", "x-request-id": "807c59b1-7633-46bb-9dda-1bb6a2f05fb2", "content-length": "152", "date": "Mon, 20 Nov 2023 07:50:34 GMT"}; value: Error { error_code: Some("DiscoveryNotYetComplete"), message: "discovery process not yet complete", request_id: "807c59b1-7633-46bb-9dda-1bb6a2f05fb2" } file = sled-agent/src/bootstrap/early_networking.rs:378
3832023-11-20T07:50:35.375ZINFOSledAgent (ServiceManager): Setting up internal-dns service file = sled-agent/src/services.rs:1577
3842023-11-20T07:50:35.559ZINFOSledAgent (ServiceManager): Initializing 1 Uplinks on Switch0 at fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:407
3852023-11-20T07:50:35.592ZINFOSledAgent (ServiceManager): Building Port Configuration file = sled-agent/src/bootstrap/early_networking.rs:533
3862023-11-20T07:50:35.592ZINFOSledAgent (ServiceManager): Checking dendrite uptime file = sled-agent/src/bootstrap/early_networking.rs:605
3872023-11-20T07:50:35.593ZINFOSledAgent (ServiceManager): Dendrite online file = sled-agent/src/bootstrap/early_networking.rs:608 uptime = 2
3882023-11-20T07:50:35.593ZINFOSledAgent (ServiceManager): Configuring boundary services loopback address on switch config = Ipv6Entry {\n addr: fd00:99::1,\n tag: "omicron",\n} file = sled-agent/src/bootstrap/early_networking.rs:429
3892023-11-20T07:50:35.593ZINFOSledAgent (ServiceManager): Configuring default uplink on switch config = PortSettings {\n links: {\n "0": LinkSettings {\n addrs: [\n 10.151.2.174,\n ],\n params: LinkCreate {\n autoneg: false,\n fec: None,\n kr: false,\n lane: Some(\n LinkId(\n 0,\n ),\n ),\n speed: Speed40G,\n },\n },\n },\n v4_routes: {\n "0.0.0.0/0": [\n RouteSettingsV4 {\n link_id: 0,\n nexthop: 10.151.2.100,\n },\n ],\n },\n v6_routes: {},\n} file = sled-agent/src/bootstrap/early_networking.rs:440
3902023-11-20T07:50:35.594ZINFOSledAgent (ServiceManager): advertising boundary services loopback address file = sled-agent/src/bootstrap/early_networking.rs:457
3912023-11-20T07:50:35.661ZINFOSledAgent (ServiceManager): Sending prefix to ddmd for advertisement DdmAdminClient = [fd00:1122:3344:101::2]:8000 file = clients/ddm-admin-client/src/lib.rs:91 prefix = Ipv6Prefix { addr: fd00:99::, len: 64 }
3922023-11-20T07:50:35.705ZINFOSledAgent (StorageManager): Received DisksChanged({}) file = sled-storage/src/manager.rs:346
3932023-11-20T07:50:35.909ZINFOSledAgent (ServiceManager): Writing ledger to /pool/int/a462a7f7-b628-40fe-80ff-4e4189e2d62b/config/.services.json.tmp file = common/src/ledger.rs:197
3942023-11-20T07:50:35.909ZINFOSledAgent (BootstrapAgentStartup): Sending prefix to ddmd for advertisement DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:91 prefix = Ipv6Prefix { addr: fd00:1122:3344:2::, len: 64 }
3952023-11-20T07:50:35.909ZINFOSledAgent (ServiceManager): Writing ledger to /pool/int/b462a7f7-b628-40fe-80ff-4e4189e2d62b/config/.services.json.tmp file = common/src/ledger.rs:197
3962023-11-20T07:50:35.909ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 10730374 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::1]:62523 req_id = 3cdcec19-56e9-4310-a066-566a66ef7f19 response_code = 204 uri = /services
3972023-11-20T07:50:35.911ZINFOSledAgent (RSS): Configuring DNS server dns_config_addr = [fd00:1122:3344:1::1]:5353 file = sled-agent/src/rack_setup/service.rs:393
3982023-11-20T07:50:35.951ZINFOSledAgent (RSS): Configuring DNS server dns_config_addr = [fd00:1122:3344:2::1]:5353 file = sled-agent/src/rack_setup/service.rs:393
3992023-11-20T07:50:36.459ZINFOSledAgent (RSS): Configuring DNS server dns_config_addr = [fd00:1122:3344:3::1]:5353 file = sled-agent/src/rack_setup/service.rs:393
4002023-11-20T07:50:36.511ZINFOSledAgent (RSS): Configured all DNS servers file = sled-agent/src/rack_setup/service.rs:429
4012023-11-20T07:50:36.511ZINFOSledAgent (RSS): Resolving switch zone addresses in DNS file = sled-agent/src/bootstrap/early_networking.rs:253
4022023-11-20T07:50:36.543ZINFOSledAgent (RSS): Querying MGS to determine switch location addr = fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:270
4032023-11-20T07:50:36.544ZINFOSledAgent (RSS): Found Switch0 at fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:295
4042023-11-20T07:50:36.544ZINFOSledAgent (RSS): Successfully looked up all expected switch zone underlay addresses addrs = {Switch0: fd00:1122:3344:101::2} file = sled-agent/src/bootstrap/early_networking.rs:187
4052023-11-20T07:50:36.573ZINFOSledAgent (RSS): sending service requests... file = sled-agent/src/rack_setup/service.rs:284
4062023-11-20T07:50:36.573ZINFOSledAgent (RSS): initializing sled services: [ServiceZoneRequest { addresses: [fd00:1122:3344:1::1], dataset: Some(DatasetRequest { id: c9a9027a-c2d6-437c-b4f4-de4f53331cd0, name: DatasetName { kind: InternalDns, pool_name: ZpoolName("oxp_14b4dc87-ab46-49fb-a4b4-d361ae214c03") }, service_address: "[fd00:1122:3344:1::1]:5353" }), id: c9a9027a-c2d6-437c-b4f4-de4f53331cd0, services: [ServiceZoneService { details: InternalDns { dns_address: "[fd00:1122:3344:1::1]:53", gz_address: fd00:1122:3344:1::2, gz_address_index: 0, http_address: "[fd00:1122:3344:1::1]:5353" }, id: c9a9027a-c2d6-437c-b4f4-de4f53331cd0 }], zone_type: InternalDns }, ServiceZoneRequest { addresses: [fd00:1122:3344:2::1], dataset: Some(DatasetRequest { id: defece7e-1f41-48b6-a56f-7f3b68d5a311, name: DatasetName { kind: InternalDns, pool_name: ZpoolName("oxp_24b4dc87-ab46-49fb-a4b4-d361ae214c03") }, service_address: "[fd00:1122:3344:2::1]:5353" }), id: defece7e-1f41-48b6-a56f-7f3b68d5a311, services: [ServiceZoneService { details: InternalDns { dns_address: "[fd00:1122:3344:2::1]:53", gz_address: fd00:1122:3344:2::2, gz_address_index: 1, http_address: "[fd00:1122:3344:2::1]:5353" }, id: defece7e-1f41-48b6-a56f-7f3b68d5a311 }], zone_type: InternalDns }, ServiceZoneRequest { addresses: [fd00:1122:3344:3::1], dataset: Some(DatasetRequest { id: 306c7298-6fe3-4323-b3ea-7a7f9075ced3, name: DatasetName { kind: InternalDns, pool_name: ZpoolName("oxp_31bd71cd-4736-4a12-a387-9b74b050396f") }, service_address: "[fd00:1122:3344:3::1]:5353" }), id: 306c7298-6fe3-4323-b3ea-7a7f9075ced3, services: [ServiceZoneService { details: InternalDns { dns_address: "[fd00:1122:3344:3::1]:53", gz_address: fd00:1122:3344:3::2, gz_address_index: 2, http_address: "[fd00:1122:3344:3::1]:5353" }, id: 306c7298-6fe3-4323-b3ea-7a7f9075ced3 }], zone_type: InternalDns }, ServiceZoneRequest { addresses: [fd00:1122:3344:101::1b], dataset: None, id: 505fd4f2-4a1e-4353-87c4-9b9b02e962ea, services: [ServiceZoneService { details: BoundaryNtp { address: "[fd00:1122:3344:101::1b]:123", dns_servers: [1.1.1.1, 9.9.9.9], domain: None, nic: NetworkInterface { id: 04c6421f-f7d5-4dd8-8ec2-99fd604b460c, ip: 172.30.3.5, kind: Service(505fd4f2-4a1e-4353-87c4-9b9b02e962ea), mac: MacAddr("A8:40:25:FF:E1:30"), name: Name("ntp-505fd4f2-4a1e-4353-87c4-9b9b02e962ea"), primary: true, slot: 0, subnet: V4(Ipv4Net("172.30.3.0/24")), vni: Vni(100) }, ntp_servers: ["0.pool.ntp.org"], snat_cfg: SourceNatConfig { first_port: 0, ip: 10.151.2.169, last_port: 16383 } }, id: 505fd4f2-4a1e-4353-87c4-9b9b02e962ea }], zone_type: Ntp }] file = sled-agent/src/rack_setup/service.rs:286
4072023-11-20T07:50:36.573ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:769 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:41300
4082023-11-20T07:50:36.573ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: c9a9027a-c2d6-437c-b4f4-de4f53331cd0, dataset_name: DatasetName { pool_name: ZpoolName { id: 14b4dc87-ab46-49fb-a4b4-d361ae214c03, kind: External }, kind: InternalDns }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } }) file = sled-storage/src/manager.rs:346
4092023-11-20T07:50:36.573ZINFOSledAgent (StorageManager): add_dataset: NewFilesystemRequest { dataset_id: c9a9027a-c2d6-437c-b4f4-de4f53331cd0, dataset_name: DatasetName { pool_name: ZpoolName { id: 14b4dc87-ab46-49fb-a4b4-d361ae214c03, kind: External }, kind: InternalDns }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } } file = sled-storage/src/manager.rs:564
4102023-11-20T07:50:36.595ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: defece7e-1f41-48b6-a56f-7f3b68d5a311, dataset_name: DatasetName { pool_name: ZpoolName { id: 24b4dc87-ab46-49fb-a4b4-d361ae214c03, kind: External }, kind: InternalDns }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } }) file = sled-storage/src/manager.rs:346
4112023-11-20T07:50:36.595ZINFOSledAgent (StorageManager): add_dataset: NewFilesystemRequest { dataset_id: defece7e-1f41-48b6-a56f-7f3b68d5a311, dataset_name: DatasetName { pool_name: ZpoolName { id: 24b4dc87-ab46-49fb-a4b4-d361ae214c03, kind: External }, kind: InternalDns }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } } file = sled-storage/src/manager.rs:564
4122023-11-20T07:50:36.614ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 306c7298-6fe3-4323-b3ea-7a7f9075ced3, dataset_name: DatasetName { pool_name: ZpoolName { id: 31bd71cd-4736-4a12-a387-9b74b050396f, kind: External }, kind: InternalDns }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } }) file = sled-storage/src/manager.rs:346
4132023-11-20T07:50:36.614ZINFOSledAgent (StorageManager): add_dataset: NewFilesystemRequest { dataset_id: 306c7298-6fe3-4323-b3ea-7a7f9075ced3, dataset_name: DatasetName { pool_name: ZpoolName { id: 31bd71cd-4736-4a12-a387-9b74b050396f, kind: External }, kind: InternalDns }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } } file = sled-storage/src/manager.rs:564
4142023-11-20T07:50:36.632ZINFOSledAgent (StorageManager): Received GetLatestResources(Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) }) file = sled-storage/src/manager.rs:346
4152023-11-20T07:50:36.633ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/a462a7f7-b628-40fe-80ff-4e4189e2d62b/config/services.json file = common/src/ledger.rs:175
4162023-11-20T07:50:36.633ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/b462a7f7-b628-40fe-80ff-4e4189e2d62b/config/services.json file = common/src/ledger.rs:175
4172023-11-20T07:50:36.633ZINFOSledAgent (StorageManager): Received GetLatestResources(Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) }) file = sled-storage/src/manager.rs:346
4182023-11-20T07:50:36.635ZINFOSledAgent (ServiceManager): Resolving switch zone addresses in DNS file = sled-agent/src/bootstrap/early_networking.rs:253
4192023-11-20T07:50:36.669ZINFOSledAgent (ServiceManager): Querying MGS to determine switch location addr = fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:270
4202023-11-20T07:50:36.670ZINFOSledAgent (ServiceManager): Found Switch0 at fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:295
4212023-11-20T07:50:36.670ZINFOSledAgent (ServiceManager): Successfully looked up all expected switch zone underlay addresses addrs = {Switch0: fd00:1122:3344:101::2} file = sled-agent/src/bootstrap/early_networking.rs:187
4222023-11-20T07:50:36.714ZINFOSledAgent (PortManager): Created OPTE port file = illumos-utils/src/opte/port_manager.rs:383 port = Port { inner: PortInner { name: "opte0", _ip: 172.30.3.5, mac: MacAddr6([168, 64, 37, 255, 225, 48]), slot: 0, vni: Vni { inner: 100 }, gateway: Gateway { mac: MacAddr6([168, 64, 37, 255, 119, 119]), ip: 172.30.3.1 }, vnic: "vopte0" } }
4232023-11-20T07:50:36.715ZINFOSledAgent (ServiceManager): creating NAT entry for service file = sled-agent/src/services.rs:893 service = ServiceZoneService { id: 505fd4f2-4a1e-4353-87c4-9b9b02e962ea, details: BoundaryNtp { address: [fd00:1122:3344:101::1b]:123, ntp_servers: ["0.pool.ntp.org"], dns_servers: [1.1.1.1, 9.9.9.9], domain: None, nic: NetworkInterface { id: 04c6421f-f7d5-4dd8-8ec2-99fd604b460c, kind: Service { id: 505fd4f2-4a1e-4353-87c4-9b9b02e962ea }, name: Name("ntp-505fd4f2-4a1e-4353-87c4-9b9b02e962ea"), ip: 172.30.3.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 225, 48])), subnet: V4(Ipv4Net(Ipv4Network { addr: 172.30.3.0, prefix: 24 })), vni: Vni(100), primary: true, slot: 0 }, snat_cfg: SourceNatConfig { ip: 10.151.2.169, first_port: 0, last_port: 16383 } } }
4242023-11-20T07:50:36.715ZINFOSledAgent (ServiceManager): no nat entry found for: V4( Ipv4Network { addr: 10.151.2.169, prefix: 32, }, ) file = clients/dpd-client/src/lib.rs:98
4252023-11-20T07:50:36.715ZINFOSledAgent (ServiceManager): creating nat entry for: V4( Ipv4Network { addr: 10.151.2.169, prefix: 32, }, ) file = clients/dpd-client/src/lib.rs:105
4262023-11-20T07:50:36.716ZINFOSledAgent (ServiceManager): creation of nat entry successful for: V4( Ipv4Network { addr: 10.151.2.169, prefix: 32, }, ) file = clients/dpd-client/src/lib.rs:133
4272023-11-20T07:50:36.717ZINFOSledAgent (StorageManager): Received GetLatestResources(Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) }) file = sled-storage/src/manager.rs:346
4282023-11-20T07:50:36.741ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_ntp_505fd4f2-4a1e-4353-87c4-9b9b02e962ea file = illumos-utils/src/zone.rs:314
4292023-11-20T07:50:36.759ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_ntp_505fd4f2-4a1e-4353-87c4-9b9b02e962ea file = illumos-utils/src/zone.rs:353
4302023-11-20T07:50:37.733ZINFOSledAgent (ServiceManager): Zone booting file = illumos-utils/src/running_zone.rs:509 zone = oxz_ntp_505fd4f2-4a1e-4353-87c4-9b9b02e962ea
4312023-11-20T07:50:37.889ZWARNSledAgent (ServiceManager): wait for service Some("oxz_ntp_505fd4f2-4a1e-4353-87c4-9b9b02e962ea") failed: Property not found. retry in 69.82267ms file = illumos-utils/src/svc.rs:36 zone = oxz_ntp_505fd4f2-4a1e-4353-87c4-9b9b02e962ea
4322023-11-20T07:50:37.972ZWARNSledAgent (ServiceManager): wait for service Some("oxz_ntp_505fd4f2-4a1e-4353-87c4-9b9b02e962ea") failed: Property not found. retry in 120.09852ms file = illumos-utils/src/svc.rs:36 zone = oxz_ntp_505fd4f2-4a1e-4353-87c4-9b9b02e962ea
4332023-11-20T07:50:38.101ZWARNSledAgent (ServiceManager): wait for service Some("oxz_ntp_505fd4f2-4a1e-4353-87c4-9b9b02e962ea") failed: Property not found. retry in 166.731685ms file = illumos-utils/src/svc.rs:36 zone = oxz_ntp_505fd4f2-4a1e-4353-87c4-9b9b02e962ea
4342023-11-20T07:50:38.278ZWARNSledAgent (ServiceManager): wait for service Some("oxz_ntp_505fd4f2-4a1e-4353-87c4-9b9b02e962ea") failed: Property not found. retry in 350.060656ms file = illumos-utils/src/svc.rs:36 zone = oxz_ntp_505fd4f2-4a1e-4353-87c4-9b9b02e962ea
4352023-11-20T07:50:38.339ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::f858:c2ff:fe1d:5103": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
4362023-11-20T07:50:38.638ZWARNSledAgent (ServiceManager): wait for service Some("oxz_ntp_505fd4f2-4a1e-4353-87c4-9b9b02e962ea") failed: Property not found. retry in 893.898642ms file = illumos-utils/src/svc.rs:36 zone = oxz_ntp_505fd4f2-4a1e-4353-87c4-9b9b02e962ea
4372023-11-20T07:50:38.884ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:769 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:58294
4382023-11-20T07:50:38.885ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 74 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:58294 req_id = 0db6820a-08aa-4849-a6b1-87e932a9d05b response_code = 200 uri = /baseboard
4392023-11-20T07:50:39.264ZWARNSledAgent: failed to register as a metric producer with Nexus error = "Metric registration error: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/metrics/producers): error trying to connect: dns error: no record found for Query { name: Name(\\"_nexus._tcp.control-plane.oxide.internal.\\"), query_type: SRV, query_class: IN }" file = sled-agent/src/sled_agent.rs:1053 retry_after = 8.495065492s sled_id = e75545d5-2741-43ba-8c56-e7f2943a5b82
4402023-11-20T07:50:39.542ZWARNSledAgent (ServiceManager): wait for service Some("oxz_ntp_505fd4f2-4a1e-4353-87c4-9b9b02e962ea") failed: Property not found. retry in 743.252704ms file = illumos-utils/src/svc.rs:36 zone = oxz_ntp_505fd4f2-4a1e-4353-87c4-9b9b02e962ea
4412023-11-20T07:50:40.295ZWARNSledAgent (ServiceManager): wait for service Some("oxz_ntp_505fd4f2-4a1e-4353-87c4-9b9b02e962ea") failed: Property not found. retry in 1.059508113s file = illumos-utils/src/svc.rs:36 zone = oxz_ntp_505fd4f2-4a1e-4353-87c4-9b9b02e962ea
4422023-11-20T07:50:41.364ZWARNSledAgent (ServiceManager): wait for service Some("oxz_ntp_505fd4f2-4a1e-4353-87c4-9b9b02e962ea") failed: Property not found. retry in 827.695081ms file = illumos-utils/src/svc.rs:36 zone = oxz_ntp_505fd4f2-4a1e-4353-87c4-9b9b02e962ea
4432023-11-20T07:50:42.201ZWARNSledAgent (ServiceManager): wait for service Some("oxz_ntp_505fd4f2-4a1e-4353-87c4-9b9b02e962ea") failed: Property not found. retry in 1.098555734s file = illumos-utils/src/svc.rs:36 zone = oxz_ntp_505fd4f2-4a1e-4353-87c4-9b9b02e962ea
4442023-11-20T07:50:43.338ZINFOSledAgent (ServiceManager): Ensuring address fd00:1122:3344:101::1b exists file = sled-agent/src/services.rs:1370
4452023-11-20T07:50:43.338ZINFOSledAgent (ServiceManager): Adding address: Static(V6(Ipv6Network { addr: fd00:1122:3344:101::1b, prefix: 64 })) file = illumos-utils/src/running_zone.rs:605 zone = oxz_ntp_505fd4f2-4a1e-4353-87c4-9b9b02e962ea
4462023-11-20T07:50:43.885ZINFOSledAgent (ServiceManager): Ensuring address fd00:1122:3344:101::1b exists - OK file = sled-agent/src/services.rs:1378
4472023-11-20T07:50:43.885ZINFOSledAgent (ServiceManager): Zone using sled underlay as gateway file = sled-agent/src/services.rs:1399
4482023-11-20T07:50:44.080ZINFOSledAgent (ServiceManager): Set up NTP service boundary=true, Servers=["0.pool.ntp.org"] file = sled-agent/src/services.rs:1931
4492023-11-20T07:50:44.107ZINFOSledAgent (ServiceManager): Ensuring address for OPTE port file = illumos-utils/src/running_zone.rs:649 zone = oxz_ntp_505fd4f2-4a1e-4353-87c4-9b9b02e962ea
4502023-11-20T07:50:49.618ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::f858:c2ff:fe1d:5103": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
4512023-11-20T07:50:49.670ZINFOSledAgent (ServiceManager): Writing ledger to /pool/int/a462a7f7-b628-40fe-80ff-4e4189e2d62b/config/.services.json.tmp file = common/src/ledger.rs:197
4522023-11-20T07:50:49.670ZINFOSledAgent (ServiceManager): Writing ledger to /pool/int/b462a7f7-b628-40fe-80ff-4e4189e2d62b/config/.services.json.tmp file = common/src/ledger.rs:197
4532023-11-20T07:50:49.670ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 13096910 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::1]:41300 req_id = a83b6afb-d392-415b-9c96-ac907adf3088 response_code = 204 uri = /services
4542023-11-20T07:50:49.673ZINFOSledAgent (RSS): Waiting for rack time synchronization file = sled-agent/src/rack_setup/service.rs:470
4552023-11-20T07:50:49.703ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:450
4562023-11-20T07:50:49.703ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:769 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:60107
4572023-11-20T07:50:49.712ZINFOSledAgent (ServiceManager): chronyc command failed: Error running command in zone 'oxz_ntp_505fd4f2-4a1e-4353-87c4-9b9b02e962ea': Command [/usr/bin/chronyc -c tracking] executed and failed with status: exit status: 1 stdout: 506 Cannot talk to daemon stderr: file = sled-agent/src/services.rs:2506
4582023-11-20T07:50:49.712ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = NTP zone not ready file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:841 latency_us = 9202 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:60107 req_id = a4f54025-8d23-4e1d-b11d-f6b342a6b20a response_code = 500 uri = /timesync
4592023-11-20T07:50:49.715ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:500
4602023-11-20T07:50:49.876ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:450
4612023-11-20T07:50:49.877ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:769 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:35651
4622023-11-20T07:50:49.884ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 7628 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:35651 req_id = 82a30b31-6fab-4032-9ffe-f976fe86fba4 response_code = 200 uri = /timesync
4632023-11-20T07:50:49.887ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 2139029761, ip_addr: ::, stratum: 10, ref_time: 1700466648.466131, correction: 0.0 } file = sled-agent/src/rack_setup/service.rs:478
4642023-11-20T07:50:49.887ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:500
4652023-11-20T07:50:50.027ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:450
4662023-11-20T07:50:50.027ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:769 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:61714
4672023-11-20T07:50:50.035ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 7671 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:61714 req_id = 3edf6878-e768-4e05-ad12-a96d87c03188 response_code = 200 uri = /timesync
4682023-11-20T07:50:50.037ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 2139029761, ip_addr: ::, stratum: 10, ref_time: 1700466648.466131, correction: 0.0 } file = sled-agent/src/rack_setup/service.rs:478
4692023-11-20T07:50:50.037ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:500
4702023-11-20T07:50:50.211ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:450
4712023-11-20T07:50:50.211ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:769 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:53698
4722023-11-20T07:50:50.220ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 9355 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:53698 req_id = dfafdddf-23bc-4656-a9c5-b6af169a114d response_code = 200 uri = /timesync
4732023-11-20T07:50:50.222ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 2139029761, ip_addr: ::, stratum: 10, ref_time: 1700466648.466131, correction: 0.0 } file = sled-agent/src/rack_setup/service.rs:478
4742023-11-20T07:50:50.222ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:500
4752023-11-20T07:50:50.440ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:450
4762023-11-20T07:50:50.440ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:769 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:47426
4772023-11-20T07:50:50.448ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 7984 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:47426 req_id = 685f29ee-7f53-4576-a4d4-100b5a70ec62 response_code = 200 uri = /timesync
4782023-11-20T07:50:50.450ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 2139029761, ip_addr: ::, stratum: 10, ref_time: 1700466648.466131, correction: 0.0 } file = sled-agent/src/rack_setup/service.rs:478
4792023-11-20T07:50:50.450ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:500
4802023-11-20T07:50:50.738ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:450
4812023-11-20T07:50:50.738ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:769 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:33432
4822023-11-20T07:50:50.746ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 8148 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:33432 req_id = f6f04169-a513-4853-9274-870897e7f13e response_code = 200 uri = /timesync
4832023-11-20T07:50:50.748ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 2139029761, ip_addr: ::, stratum: 10, ref_time: 1700466648.466131, correction: 0.0 } file = sled-agent/src/rack_setup/service.rs:478
4842023-11-20T07:50:50.748ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:500
4852023-11-20T07:50:51.121ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:450
4862023-11-20T07:50:51.121ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:769 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:33354
4872023-11-20T07:50:51.129ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 7718 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:33354 req_id = ef71b10c-73cc-48f2-9147-8d1caa81d6fe response_code = 200 uri = /timesync
4882023-11-20T07:50:51.132ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 2139029761, ip_addr: ::, stratum: 10, ref_time: 1700466648.466131, correction: 0.0 } file = sled-agent/src/rack_setup/service.rs:478
4892023-11-20T07:50:51.132ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:500
4902023-11-20T07:50:51.462ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:450
4912023-11-20T07:50:51.462ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:769 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:55488
4922023-11-20T07:50:51.471ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 8259 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:55488 req_id = 9976b5cb-6eca-4415-bedd-2db2df148f88 response_code = 200 uri = /timesync
4932023-11-20T07:50:51.473ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 2139029761, ip_addr: ::, stratum: 10, ref_time: 1700466648.466131, correction: 0.0 } file = sled-agent/src/rack_setup/service.rs:478
4942023-11-20T07:50:51.473ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:500
4952023-11-20T07:50:51.835ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:450
4962023-11-20T07:50:51.835ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:769 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:39923
4972023-11-20T07:50:51.842ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 6794 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:39923 req_id = 88d5fd81-c301-4d63-a26b-ff04931a1434 response_code = 200 uri = /timesync
4982023-11-20T07:50:51.844ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 2139029761, ip_addr: ::, stratum: 10, ref_time: 1700466648.466131, correction: 0.0 } file = sled-agent/src/rack_setup/service.rs:478
4992023-11-20T07:50:51.844ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:500
5002023-11-20T07:50:52.435ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:450
5012023-11-20T07:50:52.435ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:769 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:55835
5022023-11-20T07:50:52.443ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 8401 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:55835 req_id = 62fac8af-fd79-4912-8d96-ee2688be6c4d response_code = 200 uri = /timesync
5032023-11-20T07:50:52.446ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 2139029761, ip_addr: ::, stratum: 10, ref_time: 1700466648.466131, correction: 0.0 } file = sled-agent/src/rack_setup/service.rs:478
5042023-11-20T07:50:52.446ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:500
5052023-11-20T07:50:53.095ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:450
5062023-11-20T07:50:53.095ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:769 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:41221
5072023-11-20T07:50:53.104ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 8891 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:41221 req_id = daf5858e-08df-42a9-aab2-8117f8d7c7b4 response_code = 200 uri = /timesync
5082023-11-20T07:50:53.106ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 2139029761, ip_addr: ::, stratum: 10, ref_time: 1700466648.466131, correction: 0.0 } file = sled-agent/src/rack_setup/service.rs:478
5092023-11-20T07:50:53.106ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:500
5102023-11-20T07:50:53.791ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:450
5112023-11-20T07:50:53.791ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:769 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:56079
5122023-11-20T07:50:53.799ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 7584 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:56079 req_id = 5da8b844-e8ba-4f01-a020-e4f5b804bc97 response_code = 200 uri = /timesync
5132023-11-20T07:50:53.801ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 2139029761, ip_addr: ::, stratum: 10, ref_time: 1700466648.466131, correction: 0.0 } file = sled-agent/src/rack_setup/service.rs:478
5142023-11-20T07:50:53.801ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:500
5152023-11-20T07:50:54.346ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:450
5162023-11-20T07:50:54.347ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:769 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:60042
5172023-11-20T07:50:54.355ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 8599 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:60042 req_id = d1e6fee6-2837-4da9-b2bf-5a69af40fe56 response_code = 200 uri = /timesync
5182023-11-20T07:50:54.357ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 2139029761, ip_addr: ::, stratum: 10, ref_time: 1700466648.466131, correction: 0.0 } file = sled-agent/src/rack_setup/service.rs:478
5192023-11-20T07:50:54.357ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:500
5202023-11-20T07:50:54.623ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::f858:c2ff:fe1d:5103": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
5212023-11-20T07:50:55.536ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:450
5222023-11-20T07:50:55.536ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:769 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:36656
5232023-11-20T07:50:55.545ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 8909 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:36656 req_id = 24e41f4a-f81a-439f-9724-108f55304548 response_code = 200 uri = /timesync
5242023-11-20T07:50:55.547ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 2139029761, ip_addr: ::, stratum: 10, ref_time: 1700466648.466131, correction: 0.015687648 } file = sled-agent/src/rack_setup/service.rs:478
5252023-11-20T07:50:55.547ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:500
5262023-11-20T07:50:56.526ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:450
5272023-11-20T07:50:56.526ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:769 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:64702
5282023-11-20T07:50:56.534ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 8371 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:64702 req_id = 897d6407-17ad-4b17-a335-8aabc15cdff3 response_code = 200 uri = /timesync
5292023-11-20T07:50:56.536ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 2139029761, ip_addr: ::, stratum: 10, ref_time: 1700466648.466131, correction: 0.013015469 } file = sled-agent/src/rack_setup/service.rs:478
5302023-11-20T07:50:56.536ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:500
5312023-11-20T07:50:57.289ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:450
5322023-11-20T07:50:57.289ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:769 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:47521
5332023-11-20T07:50:57.297ZINFOSledAgent (ServiceManager): Setting boot time to 1700466657.29762163s file = sled-agent/src/services.rs:2405
5342023-11-20T07:50:57.303ZWARNSledAgent (ServiceManager): Updating /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_internal_dns_306c7298-6fe3-4323-b3ea-7a7f9075ced3/root/var/adm/utmpx failed: Command [/usr/platform/oxide/bin/tmpx 1700466657 /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_internal_dns_306c7298-6fe3-4323-b3ea-7a7f9075ced3/root/var/adm/utmpx] executed and failed with status: exit status: 1 stdout: stderr: tmpx: Invalid path: '/pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_internal_dns_306c7298-6fe3-4323-b3ea-7a7f9075ced3/root/var/adm/utmpx' (see utmpxname(3C)) file = sled-agent/src/services.rs:2422
5352023-11-20T07:50:57.308ZWARNSledAgent (ServiceManager): Updating /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_internal_dns_306c7298-6fe3-4323-b3ea-7a7f9075ced3/root/var/adm/wtmpx failed: Command [/usr/platform/oxide/bin/tmpx 1700466657 /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_internal_dns_306c7298-6fe3-4323-b3ea-7a7f9075ced3/root/var/adm/wtmpx] executed and failed with status: exit status: 1 stdout: stderr: tmpx: Invalid path: '/pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_internal_dns_306c7298-6fe3-4323-b3ea-7a7f9075ced3/root/var/adm/wtmpx' (see utmpxname(3C)) file = sled-agent/src/services.rs:2422
5362023-11-20T07:50:57.314ZWARNSledAgent (ServiceManager): Updating /pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_internal_dns_c9a9027a-c2d6-437c-b4f4-de4f53331cd0/root/var/adm/utmpx failed: Command [/usr/platform/oxide/bin/tmpx 1700466657 /pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_internal_dns_c9a9027a-c2d6-437c-b4f4-de4f53331cd0/root/var/adm/utmpx] executed and failed with status: exit status: 1 stdout: stderr: tmpx: Invalid path: '/pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_internal_dns_c9a9027a-c2d6-437c-b4f4-de4f53331cd0/root/var/adm/utmpx' (see utmpxname(3C)) file = sled-agent/src/services.rs:2422
5372023-11-20T07:50:57.319ZWARNSledAgent (ServiceManager): Updating /pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_internal_dns_c9a9027a-c2d6-437c-b4f4-de4f53331cd0/root/var/adm/wtmpx failed: Command [/usr/platform/oxide/bin/tmpx 1700466657 /pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_internal_dns_c9a9027a-c2d6-437c-b4f4-de4f53331cd0/root/var/adm/wtmpx] executed and failed with status: exit status: 1 stdout: stderr: tmpx: Invalid path: '/pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_internal_dns_c9a9027a-c2d6-437c-b4f4-de4f53331cd0/root/var/adm/wtmpx' (see utmpxname(3C)) file = sled-agent/src/services.rs:2422
5382023-11-20T07:50:57.325ZWARNSledAgent (ServiceManager): Updating /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_internal_dns_defece7e-1f41-48b6-a56f-7f3b68d5a311/root/var/adm/utmpx failed: Command [/usr/platform/oxide/bin/tmpx 1700466657 /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_internal_dns_defece7e-1f41-48b6-a56f-7f3b68d5a311/root/var/adm/utmpx] executed and failed with status: exit status: 1 stdout: stderr: tmpx: Invalid path: '/pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_internal_dns_defece7e-1f41-48b6-a56f-7f3b68d5a311/root/var/adm/utmpx' (see utmpxname(3C)) file = sled-agent/src/services.rs:2422
5392023-11-20T07:50:57.331ZWARNSledAgent (ServiceManager): Updating /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_internal_dns_defece7e-1f41-48b6-a56f-7f3b68d5a311/root/var/adm/wtmpx failed: Command [/usr/platform/oxide/bin/tmpx 1700466657 /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_internal_dns_defece7e-1f41-48b6-a56f-7f3b68d5a311/root/var/adm/wtmpx] executed and failed with status: exit status: 1 stdout: stderr: tmpx: Invalid path: '/pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_internal_dns_defece7e-1f41-48b6-a56f-7f3b68d5a311/root/var/adm/wtmpx' (see utmpxname(3C)) file = sled-agent/src/services.rs:2422
5402023-11-20T07:50:57.336ZWARNSledAgent (ServiceManager): Updating /pool/ext/e4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_ntp_505fd4f2-4a1e-4353-87c4-9b9b02e962ea/root/var/adm/utmpx failed: Command [/usr/platform/oxide/bin/tmpx 1700466657 /pool/ext/e4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_ntp_505fd4f2-4a1e-4353-87c4-9b9b02e962ea/root/var/adm/utmpx] executed and failed with status: exit status: 1 stdout: stderr: tmpx: Invalid path: '/pool/ext/e4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_ntp_505fd4f2-4a1e-4353-87c4-9b9b02e962ea/root/var/adm/utmpx' (see utmpxname(3C)) file = sled-agent/src/services.rs:2422
5412023-11-20T07:50:57.342ZWARNSledAgent (ServiceManager): Updating /pool/ext/e4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_ntp_505fd4f2-4a1e-4353-87c4-9b9b02e962ea/root/var/adm/wtmpx failed: Command [/usr/platform/oxide/bin/tmpx 1700466657 /pool/ext/e4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_ntp_505fd4f2-4a1e-4353-87c4-9b9b02e962ea/root/var/adm/wtmpx] executed and failed with status: exit status: 1 stdout: stderr: tmpx: Invalid path: '/pool/ext/e4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_ntp_505fd4f2-4a1e-4353-87c4-9b9b02e962ea/root/var/adm/wtmpx' (see utmpxname(3C)) file = sled-agent/src/services.rs:2422
5422023-11-20T07:50:57.347ZINFOSledAgent (ServiceManager): Updated /var/adm/utmpx file = sled-agent/src/services.rs:2425
5432023-11-20T07:50:57.352ZINFOSledAgent (ServiceManager): Updated /var/adm/wtmpx file = sled-agent/src/services.rs:2425
5442023-11-20T07:50:57.352ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 62868 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:47521 req_id = 3e768d06-657a-46c4-976c-ceb9c409720a response_code = 200 uri = /timesync
5452023-11-20T07:50:57.354ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: true, ref_id: 2728380539, ip_addr: 162.159.200.123, stratum: 4, ref_time: 1700466656.3894222, correction: 0.011225877 } file = sled-agent/src/rack_setup/service.rs:478
5462023-11-20T07:50:57.354ZINFOSledAgent (RSS): Finished setting up Internal DNS and NTP file = sled-agent/src/rack_setup/service.rs:975
5472023-11-20T07:50:57.384ZINFOSledAgent (RSS): sending service requests... file = sled-agent/src/rack_setup/service.rs:284
5482023-11-20T07:50:57.384ZINFOSledAgent (RSS): initializing sled services: [ServiceZoneRequest { addresses: [fd00:1122:3344:1::1], dataset: Some(DatasetRequest { id: c9a9027a-c2d6-437c-b4f4-de4f53331cd0, name: DatasetName { kind: InternalDns, pool_name: ZpoolName("oxp_14b4dc87-ab46-49fb-a4b4-d361ae214c03") }, service_address: "[fd00:1122:3344:1::1]:5353" }), id: c9a9027a-c2d6-437c-b4f4-de4f53331cd0, services: [ServiceZoneService { details: InternalDns { dns_address: "[fd00:1122:3344:1::1]:53", gz_address: fd00:1122:3344:1::2, gz_address_index: 0, http_address: "[fd00:1122:3344:1::1]:5353" }, id: c9a9027a-c2d6-437c-b4f4-de4f53331cd0 }], zone_type: InternalDns }, ServiceZoneRequest { addresses: [fd00:1122:3344:2::1], dataset: Some(DatasetRequest { id: defece7e-1f41-48b6-a56f-7f3b68d5a311, name: DatasetName { kind: InternalDns, pool_name: ZpoolName("oxp_24b4dc87-ab46-49fb-a4b4-d361ae214c03") }, service_address: "[fd00:1122:3344:2::1]:5353" }), id: defece7e-1f41-48b6-a56f-7f3b68d5a311, services: [ServiceZoneService { details: InternalDns { dns_address: "[fd00:1122:3344:2::1]:53", gz_address: fd00:1122:3344:2::2, gz_address_index: 1, http_address: "[fd00:1122:3344:2::1]:5353" }, id: defece7e-1f41-48b6-a56f-7f3b68d5a311 }], zone_type: InternalDns }, ServiceZoneRequest { addresses: [fd00:1122:3344:3::1], dataset: Some(DatasetRequest { id: 306c7298-6fe3-4323-b3ea-7a7f9075ced3, name: DatasetName { kind: InternalDns, pool_name: ZpoolName("oxp_31bd71cd-4736-4a12-a387-9b74b050396f") }, service_address: "[fd00:1122:3344:3::1]:5353" }), id: 306c7298-6fe3-4323-b3ea-7a7f9075ced3, services: [ServiceZoneService { details: InternalDns { dns_address: "[fd00:1122:3344:3::1]:53", gz_address: fd00:1122:3344:3::2, gz_address_index: 2, http_address: "[fd00:1122:3344:3::1]:5353" }, id: 306c7298-6fe3-4323-b3ea-7a7f9075ced3 }], zone_type: InternalDns }, ServiceZoneRequest { addresses: [fd00:1122:3344:101::3], dataset: Some(DatasetRequest { id: a3323bbb-65d3-472d-a293-6c510636cffb, name: DatasetName { kind: CockroachDb, pool_name: ZpoolName("oxp_14b4dc87-ab46-49fb-a4b4-d361ae214c03") }, service_address: "[fd00:1122:3344:101::3]:32221" }), id: a3323bbb-65d3-472d-a293-6c510636cffb, services: [ServiceZoneService { details: CockroachDb { address: "[fd00:1122:3344:101::3]:32221" }, id: a3323bbb-65d3-472d-a293-6c510636cffb }], zone_type: CockroachDb }, ServiceZoneRequest { addresses: [fd00:1122:3344:101::4], dataset: Some(DatasetRequest { id: 33c7324a-3441-45ea-8d79-212b82858480, name: DatasetName { kind: CockroachDb, pool_name: ZpoolName("oxp_24b4dc87-ab46-49fb-a4b4-d361ae214c03") }, service_address: "[fd00:1122:3344:101::4]:32221" }), id: 33c7324a-3441-45ea-8d79-212b82858480, services: [ServiceZoneService { details: CockroachDb { address: "[fd00:1122:3344:101::4]:32221" }, id: 33c7324a-3441-45ea-8d79-212b82858480 }], zone_type: CockroachDb }, ServiceZoneRequest { addresses: [fd00:1122:3344:101::5], dataset: Some(DatasetRequest { id: f561426b-04e6-4e63-b481-1b397d5231d4, name: DatasetName { kind: CockroachDb, pool_name: ZpoolName("oxp_31bd71cd-4736-4a12-a387-9b74b050396f") }, service_address: "[fd00:1122:3344:101::5]:32221" }), id: f561426b-04e6-4e63-b481-1b397d5231d4, services: [ServiceZoneService { details: CockroachDb { address: "[fd00:1122:3344:101::5]:32221" }, id: f561426b-04e6-4e63-b481-1b397d5231d4 }], zone_type: CockroachDb }, ServiceZoneRequest { addresses: [fd00:1122:3344:101::6], dataset: Some(DatasetRequest { id: da996e81-243f-4d6a-9a19-416683cfc273, name: DatasetName { kind: CockroachDb, pool_name: ZpoolName("oxp_616b26df-e62a-4c68-b506-f4a923d8aaf7") }, service_address: "[fd00:1122:3344:101::6]:32221" }), id: da996e81-243f-4d6a-9a19-416683cfc273, services: [ServiceZoneService { details: CockroachDb { address: "[fd00:1122:3344:101::6]:32221" }, id: da996e81-243f-4d6a-9a19-416683cfc273 }], zone_type: CockroachDb }, ServiceZoneRequest { addresses: [fd00:1122:3344:101::7], dataset: Some(DatasetRequest { id: a09c9ef2-b44c-46b9-8f18-372e76dd8249, name: DatasetName { kind: CockroachDb, pool_name: ZpoolName("oxp_cd70d7f6-2354-4bf2-8012-55bf9eaf7930") }, service_address: "[fd00:1122:3344:101::7]:32221" }), id: a09c9ef2-b44c-46b9-8f18-372e76dd8249, services: [ServiceZoneService { details: CockroachDb { address: "[fd00:1122:3344:101::7]:32221" }, id: a09c9ef2-b44c-46b9-8f18-372e76dd8249 }], zone_type: CockroachDb }, ServiceZoneRequest { addresses: [fd00:1122:3344:101::1b], dataset: None, id: 505fd4f2-4a1e-4353-87c4-9b9b02e962ea, services: [ServiceZoneService { details: BoundaryNtp { address: "[fd00:1122:3344:101::1b]:123", dns_servers: [1.1.1.1, 9.9.9.9], domain: None, nic: NetworkInterface { id: 04c6421f-f7d5-4dd8-8ec2-99fd604b460c, ip: 172.30.3.5, kind: Service(505fd4f2-4a1e-4353-87c4-9b9b02e962ea), mac: MacAddr("A8:40:25:FF:E1:30"), name: Name("ntp-505fd4f2-4a1e-4353-87c4-9b9b02e962ea"), primary: true, slot: 0, subnet: V4(Ipv4Net("172.30.3.0/24")), vni: Vni(100) }, ntp_servers: ["0.pool.ntp.org"], snat_cfg: SourceNatConfig { first_port: 0, ip: 10.151.2.169, last_port: 16383 } }, id: 505fd4f2-4a1e-4353-87c4-9b9b02e962ea }], zone_type: Ntp }] file = sled-agent/src/rack_setup/service.rs:286
5492023-11-20T07:50:57.384ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:769 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:38094
5502023-11-20T07:50:57.385ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: c9a9027a-c2d6-437c-b4f4-de4f53331cd0, dataset_name: DatasetName { pool_name: ZpoolName { id: 14b4dc87-ab46-49fb-a4b4-d361ae214c03, kind: External }, kind: InternalDns }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } }) file = sled-storage/src/manager.rs:346
5512023-11-20T07:50:57.385ZINFOSledAgent (StorageManager): add_dataset: NewFilesystemRequest { dataset_id: c9a9027a-c2d6-437c-b4f4-de4f53331cd0, dataset_name: DatasetName { pool_name: ZpoolName { id: 14b4dc87-ab46-49fb-a4b4-d361ae214c03, kind: External }, kind: InternalDns }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } } file = sled-storage/src/manager.rs:564
5522023-11-20T07:50:57.407ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: defece7e-1f41-48b6-a56f-7f3b68d5a311, dataset_name: DatasetName { pool_name: ZpoolName { id: 24b4dc87-ab46-49fb-a4b4-d361ae214c03, kind: External }, kind: InternalDns }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } }) file = sled-storage/src/manager.rs:346
5532023-11-20T07:50:57.407ZINFOSledAgent (StorageManager): add_dataset: NewFilesystemRequest { dataset_id: defece7e-1f41-48b6-a56f-7f3b68d5a311, dataset_name: DatasetName { pool_name: ZpoolName { id: 24b4dc87-ab46-49fb-a4b4-d361ae214c03, kind: External }, kind: InternalDns }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } } file = sled-storage/src/manager.rs:564
5542023-11-20T07:50:57.424ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 306c7298-6fe3-4323-b3ea-7a7f9075ced3, dataset_name: DatasetName { pool_name: ZpoolName { id: 31bd71cd-4736-4a12-a387-9b74b050396f, kind: External }, kind: InternalDns }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } }) file = sled-storage/src/manager.rs:346
5552023-11-20T07:50:57.424ZINFOSledAgent (StorageManager): add_dataset: NewFilesystemRequest { dataset_id: 306c7298-6fe3-4323-b3ea-7a7f9075ced3, dataset_name: DatasetName { pool_name: ZpoolName { id: 31bd71cd-4736-4a12-a387-9b74b050396f, kind: External }, kind: InternalDns }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } } file = sled-storage/src/manager.rs:564
5562023-11-20T07:50:57.440ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: a3323bbb-65d3-472d-a293-6c510636cffb, dataset_name: DatasetName { pool_name: ZpoolName { id: 14b4dc87-ab46-49fb-a4b4-d361ae214c03, kind: External }, kind: CockroachDb }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } }) file = sled-storage/src/manager.rs:346
5572023-11-20T07:50:57.440ZINFOSledAgent (StorageManager): add_dataset: NewFilesystemRequest { dataset_id: a3323bbb-65d3-472d-a293-6c510636cffb, dataset_name: DatasetName { pool_name: ZpoolName { id: 14b4dc87-ab46-49fb-a4b4-d361ae214c03, kind: External }, kind: CockroachDb }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } } file = sled-storage/src/manager.rs:564
5582023-11-20T07:50:57.488ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 33c7324a-3441-45ea-8d79-212b82858480, dataset_name: DatasetName { pool_name: ZpoolName { id: 24b4dc87-ab46-49fb-a4b4-d361ae214c03, kind: External }, kind: CockroachDb }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } }) file = sled-storage/src/manager.rs:346
5592023-11-20T07:50:57.489ZINFOSledAgent (StorageManager): add_dataset: NewFilesystemRequest { dataset_id: 33c7324a-3441-45ea-8d79-212b82858480, dataset_name: DatasetName { pool_name: ZpoolName { id: 24b4dc87-ab46-49fb-a4b4-d361ae214c03, kind: External }, kind: CockroachDb }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } } file = sled-storage/src/manager.rs:564
5602023-11-20T07:50:57.543ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: f561426b-04e6-4e63-b481-1b397d5231d4, dataset_name: DatasetName { pool_name: ZpoolName { id: 31bd71cd-4736-4a12-a387-9b74b050396f, kind: External }, kind: CockroachDb }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } }) file = sled-storage/src/manager.rs:346
5612023-11-20T07:50:57.543ZINFOSledAgent (StorageManager): add_dataset: NewFilesystemRequest { dataset_id: f561426b-04e6-4e63-b481-1b397d5231d4, dataset_name: DatasetName { pool_name: ZpoolName { id: 31bd71cd-4736-4a12-a387-9b74b050396f, kind: External }, kind: CockroachDb }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } } file = sled-storage/src/manager.rs:564
5622023-11-20T07:50:57.593ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: da996e81-243f-4d6a-9a19-416683cfc273, dataset_name: DatasetName { pool_name: ZpoolName { id: 616b26df-e62a-4c68-b506-f4a923d8aaf7, kind: External }, kind: CockroachDb }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } }) file = sled-storage/src/manager.rs:346
5632023-11-20T07:50:57.593ZINFOSledAgent (StorageManager): add_dataset: NewFilesystemRequest { dataset_id: da996e81-243f-4d6a-9a19-416683cfc273, dataset_name: DatasetName { pool_name: ZpoolName { id: 616b26df-e62a-4c68-b506-f4a923d8aaf7, kind: External }, kind: CockroachDb }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } } file = sled-storage/src/manager.rs:564
5642023-11-20T07:50:57.643ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: a09c9ef2-b44c-46b9-8f18-372e76dd8249, dataset_name: DatasetName { pool_name: ZpoolName { id: cd70d7f6-2354-4bf2-8012-55bf9eaf7930, kind: External }, kind: CockroachDb }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } }) file = sled-storage/src/manager.rs:346
5652023-11-20T07:50:57.643ZINFOSledAgent (StorageManager): add_dataset: NewFilesystemRequest { dataset_id: a09c9ef2-b44c-46b9-8f18-372e76dd8249, dataset_name: DatasetName { pool_name: ZpoolName { id: cd70d7f6-2354-4bf2-8012-55bf9eaf7930, kind: External }, kind: CockroachDb }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } } file = sled-storage/src/manager.rs:564
5662023-11-20T07:50:57.691ZINFOSledAgent (StorageManager): Received GetLatestResources(Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) }) file = sled-storage/src/manager.rs:346
5672023-11-20T07:50:57.691ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/a462a7f7-b628-40fe-80ff-4e4189e2d62b/config/services.json file = common/src/ledger.rs:175
5682023-11-20T07:50:57.691ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/b462a7f7-b628-40fe-80ff-4e4189e2d62b/config/services.json file = common/src/ledger.rs:175
5692023-11-20T07:50:57.691ZINFOSledAgent (StorageManager): Received GetLatestResources(Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) }) file = sled-storage/src/manager.rs:346
5702023-11-20T07:50:57.698ZINFOSledAgent (StorageManager): Received GetLatestResources(Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) }) file = sled-storage/src/manager.rs:346
5712023-11-20T07:50:57.698ZINFOSledAgent (StorageManager): Received GetLatestResources(Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) }) file = sled-storage/src/manager.rs:346
5722023-11-20T07:50:57.698ZINFOSledAgent (StorageManager): Received GetLatestResources(Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) }) file = sled-storage/src/manager.rs:346
5732023-11-20T07:50:57.698ZINFOSledAgent (StorageManager): Received GetLatestResources(Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) }) file = sled-storage/src/manager.rs:346
5742023-11-20T07:50:57.698ZINFOSledAgent (StorageManager): Received GetLatestResources(Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) }) file = sled-storage/src/manager.rs:346
5752023-11-20T07:50:57.791ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_cockroachdb_da996e81-243f-4d6a-9a19-416683cfc273 file = illumos-utils/src/zone.rs:314
5762023-11-20T07:50:57.795ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_cockroachdb_33c7324a-3441-45ea-8d79-212b82858480 file = illumos-utils/src/zone.rs:314
5772023-11-20T07:50:57.799ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_cockroachdb_a3323bbb-65d3-472d-a293-6c510636cffb file = illumos-utils/src/zone.rs:314
5782023-11-20T07:50:57.804ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_cockroachdb_f561426b-04e6-4e63-b481-1b397d5231d4 file = illumos-utils/src/zone.rs:314
5792023-11-20T07:50:57.809ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_cockroachdb_a09c9ef2-b44c-46b9-8f18-372e76dd8249 file = illumos-utils/src/zone.rs:314
5802023-11-20T07:50:57.813ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_cockroachdb_da996e81-243f-4d6a-9a19-416683cfc273 file = illumos-utils/src/zone.rs:353
5812023-11-20T07:50:57.818ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_cockroachdb_33c7324a-3441-45ea-8d79-212b82858480 file = illumos-utils/src/zone.rs:353
5822023-11-20T07:50:57.822ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_cockroachdb_a3323bbb-65d3-472d-a293-6c510636cffb file = illumos-utils/src/zone.rs:353
5832023-11-20T07:50:57.827ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_cockroachdb_f561426b-04e6-4e63-b481-1b397d5231d4 file = illumos-utils/src/zone.rs:353
5842023-11-20T07:50:57.831ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_cockroachdb_a09c9ef2-b44c-46b9-8f18-372e76dd8249 file = illumos-utils/src/zone.rs:353
5852023-11-20T07:50:59.637ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::f858:c2ff:fe1d:5103": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
5862023-11-20T07:51:01.015ZINFOSledAgent (ServiceManager): Profile for oxz_cockroachdb_33c7324a-3441-45ea-8d79-212b82858480: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/cockroachdb"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value="oxControlService6"/> <propval type="astring" name="gateway" value="fd00:1122:3344:101::1"/> <propval type="astring" name="listen_addr" value="fd00:1122:3344:101::4"/> <propval type="astring" name="listen_port" value="32221"/> <propval type="astring" name="store" value="/data"/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/install"> <property_group type="application" name="install_props"> <property type="net_address" name="nameserver"> <net_address_list> <value_node value="fd00:1122:3344:3::1" /> <value_node value="fd00:1122:3344:1::1" /> <value_node value="fd00:1122:3344:2::1" /> </net_address_list> </property> </property_group> <instance enabled="true" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
5872023-11-20T07:51:01.015ZINFOSledAgent (ServiceManager): Zone booting file = illumos-utils/src/running_zone.rs:509 zone = oxz_cockroachdb_33c7324a-3441-45ea-8d79-212b82858480
5882023-11-20T07:51:01.033ZINFOSledAgent (ServiceManager): Profile for oxz_cockroachdb_f561426b-04e6-4e63-b481-1b397d5231d4: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/cockroachdb"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value="oxControlService8"/> <propval type="astring" name="gateway" value="fd00:1122:3344:101::1"/> <propval type="astring" name="listen_addr" value="fd00:1122:3344:101::5"/> <propval type="astring" name="listen_port" value="32221"/> <propval type="astring" name="store" value="/data"/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/install"> <property_group type="application" name="install_props"> <property type="net_address" name="nameserver"> <net_address_list> <value_node value="fd00:1122:3344:3::1" /> <value_node value="fd00:1122:3344:1::1" /> <value_node value="fd00:1122:3344:2::1" /> </net_address_list> </property> </property_group> <instance enabled="true" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
5892023-11-20T07:51:01.033ZINFOSledAgent (ServiceManager): Zone booting file = illumos-utils/src/running_zone.rs:509 zone = oxz_cockroachdb_f561426b-04e6-4e63-b481-1b397d5231d4
5902023-11-20T07:51:01.154ZINFOSledAgent (ServiceManager): Profile for oxz_cockroachdb_a3323bbb-65d3-472d-a293-6c510636cffb: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/cockroachdb"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value="oxControlService7"/> <propval type="astring" name="gateway" value="fd00:1122:3344:101::1"/> <propval type="astring" name="listen_addr" value="fd00:1122:3344:101::3"/> <propval type="astring" name="listen_port" value="32221"/> <propval type="astring" name="store" value="/data"/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/install"> <property_group type="application" name="install_props"> <property type="net_address" name="nameserver"> <net_address_list> <value_node value="fd00:1122:3344:3::1" /> <value_node value="fd00:1122:3344:1::1" /> <value_node value="fd00:1122:3344:2::1" /> </net_address_list> </property> </property_group> <instance enabled="true" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
5912023-11-20T07:51:01.154ZINFOSledAgent (ServiceManager): Zone booting file = illumos-utils/src/running_zone.rs:509 zone = oxz_cockroachdb_a3323bbb-65d3-472d-a293-6c510636cffb
5922023-11-20T07:51:01.166ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_33c7324a-3441-45ea-8d79-212b82858480") failed: Property not found. retry in 37.962617ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_33c7324a-3441-45ea-8d79-212b82858480
5932023-11-20T07:51:01.173ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_f561426b-04e6-4e63-b481-1b397d5231d4") failed: Property not found. retry in 32.638125ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_f561426b-04e6-4e63-b481-1b397d5231d4
5942023-11-20T07:51:01.212ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_33c7324a-3441-45ea-8d79-212b82858480") failed: Property not found. retry in 125.013986ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_33c7324a-3441-45ea-8d79-212b82858480
5952023-11-20T07:51:01.219ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_f561426b-04e6-4e63-b481-1b397d5231d4") failed: Property not found. retry in 141.361544ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_f561426b-04e6-4e63-b481-1b397d5231d4
5962023-11-20T07:51:01.220ZINFOSledAgent (ServiceManager): Profile for oxz_cockroachdb_a09c9ef2-b44c-46b9-8f18-372e76dd8249: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/cockroachdb"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value="oxControlService9"/> <propval type="astring" name="gateway" value="fd00:1122:3344:101::1"/> <propval type="astring" name="listen_addr" value="fd00:1122:3344:101::7"/> <propval type="astring" name="listen_port" value="32221"/> <propval type="astring" name="store" value="/data"/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/install"> <property_group type="application" name="install_props"> <property type="net_address" name="nameserver"> <net_address_list> <value_node value="fd00:1122:3344:3::1" /> <value_node value="fd00:1122:3344:1::1" /> <value_node value="fd00:1122:3344:2::1" /> </net_address_list> </property> </property_group> <instance enabled="true" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
5972023-11-20T07:51:01.220ZINFOSledAgent (ServiceManager): Zone booting file = illumos-utils/src/running_zone.rs:509 zone = oxz_cockroachdb_a09c9ef2-b44c-46b9-8f18-372e76dd8249
5982023-11-20T07:51:01.423ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_33c7324a-3441-45ea-8d79-212b82858480") failed: Property not found. retry in 148.229667ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_33c7324a-3441-45ea-8d79-212b82858480
5992023-11-20T07:51:01.456ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_f561426b-04e6-4e63-b481-1b397d5231d4") failed: Property not found. retry in 192.471467ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_f561426b-04e6-4e63-b481-1b397d5231d4
6002023-11-20T07:51:01.461ZINFOSledAgent (ServiceManager): Profile for oxz_cockroachdb_da996e81-243f-4d6a-9a19-416683cfc273: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/cockroachdb"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value="oxControlService5"/> <propval type="astring" name="gateway" value="fd00:1122:3344:101::1"/> <propval type="astring" name="listen_addr" value="fd00:1122:3344:101::6"/> <propval type="astring" name="listen_port" value="32221"/> <propval type="astring" name="store" value="/data"/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/install"> <property_group type="application" name="install_props"> <property type="net_address" name="nameserver"> <net_address_list> <value_node value="fd00:1122:3344:3::1" /> <value_node value="fd00:1122:3344:1::1" /> <value_node value="fd00:1122:3344:2::1" /> </net_address_list> </property> </property_group> <instance enabled="true" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
6012023-11-20T07:51:01.461ZINFOSledAgent (ServiceManager): Zone booting file = illumos-utils/src/running_zone.rs:509 zone = oxz_cockroachdb_da996e81-243f-4d6a-9a19-416683cfc273
6022023-11-20T07:51:01.652ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_33c7324a-3441-45ea-8d79-212b82858480") failed: Property not found. retry in 464.001105ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_33c7324a-3441-45ea-8d79-212b82858480
6032023-11-20T07:51:01.806ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_f561426b-04e6-4e63-b481-1b397d5231d4") failed: Property not found. retry in 498.489359ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_f561426b-04e6-4e63-b481-1b397d5231d4
6042023-11-20T07:51:01.856ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_a3323bbb-65d3-472d-a293-6c510636cffb") failed: Property not found. retry in 60.418134ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_a3323bbb-65d3-472d-a293-6c510636cffb
6052023-11-20T07:51:01.961ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_a3323bbb-65d3-472d-a293-6c510636cffb") failed: Property not found. retry in 94.026068ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_a3323bbb-65d3-472d-a293-6c510636cffb
6062023-11-20T07:51:01.998ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_a09c9ef2-b44c-46b9-8f18-372e76dd8249") failed: Property not found. retry in 66.029693ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_a09c9ef2-b44c-46b9-8f18-372e76dd8249
6072023-11-20T07:51:02.230ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_da996e81-243f-4d6a-9a19-416683cfc273") failed: Property not found. retry in 40.395559ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_da996e81-243f-4d6a-9a19-416683cfc273
6082023-11-20T07:51:02.450ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_33c7324a-3441-45ea-8d79-212b82858480") failed: Property not found. retry in 987.750735ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_33c7324a-3441-45ea-8d79-212b82858480
6092023-11-20T07:51:02.524ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_a3323bbb-65d3-472d-a293-6c510636cffb") failed: Property not found. retry in 285.968343ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_a3323bbb-65d3-472d-a293-6c510636cffb
6102023-11-20T07:51:02.549ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_f561426b-04e6-4e63-b481-1b397d5231d4") failed: Property not found. retry in 659.596646ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_f561426b-04e6-4e63-b481-1b397d5231d4
6112023-11-20T07:51:02.622ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_a09c9ef2-b44c-46b9-8f18-372e76dd8249") failed: Property not found. retry in 117.338871ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_a09c9ef2-b44c-46b9-8f18-372e76dd8249
6122023-11-20T07:51:02.636ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_da996e81-243f-4d6a-9a19-416683cfc273") failed: Property not found. retry in 121.998068ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_da996e81-243f-4d6a-9a19-416683cfc273
6132023-11-20T07:51:02.759ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_a09c9ef2-b44c-46b9-8f18-372e76dd8249") failed: Property not found. retry in 248.376763ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_a09c9ef2-b44c-46b9-8f18-372e76dd8249
6142023-11-20T07:51:02.806ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_da996e81-243f-4d6a-9a19-416683cfc273") failed: Property not found. retry in 116.879933ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_da996e81-243f-4d6a-9a19-416683cfc273
6152023-11-20T07:51:02.843ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_a3323bbb-65d3-472d-a293-6c510636cffb") failed: Property not found. retry in 268.680718ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_a3323bbb-65d3-472d-a293-6c510636cffb
6162023-11-20T07:51:02.950ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_da996e81-243f-4d6a-9a19-416683cfc273") failed: Property not found. retry in 599.763208ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_da996e81-243f-4d6a-9a19-416683cfc273
6172023-11-20T07:51:03.028ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_a09c9ef2-b44c-46b9-8f18-372e76dd8249") failed: Property not found. retry in 257.948962ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_a09c9ef2-b44c-46b9-8f18-372e76dd8249
6182023-11-20T07:51:03.145ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_a3323bbb-65d3-472d-a293-6c510636cffb") failed: Property not found. retry in 624.090078ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_a3323bbb-65d3-472d-a293-6c510636cffb
6192023-11-20T07:51:03.266ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_f561426b-04e6-4e63-b481-1b397d5231d4") failed: Property not found. retry in 592.297571ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_f561426b-04e6-4e63-b481-1b397d5231d4
6202023-11-20T07:51:03.307ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_a09c9ef2-b44c-46b9-8f18-372e76dd8249") failed: Property not found. retry in 720.590634ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_a09c9ef2-b44c-46b9-8f18-372e76dd8249
6212023-11-20T07:51:03.450ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_33c7324a-3441-45ea-8d79-212b82858480") failed: Property not found. retry in 1.150369978s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_33c7324a-3441-45ea-8d79-212b82858480
6222023-11-20T07:51:03.562ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_da996e81-243f-4d6a-9a19-416683cfc273") failed: Property not found. retry in 620.384579ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_da996e81-243f-4d6a-9a19-416683cfc273
6232023-11-20T07:51:03.778ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_a3323bbb-65d3-472d-a293-6c510636cffb") failed: Property not found. retry in 1.427474733s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_a3323bbb-65d3-472d-a293-6c510636cffb
6242023-11-20T07:51:03.869ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_f561426b-04e6-4e63-b481-1b397d5231d4") failed: Property not found. retry in 852.192122ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_f561426b-04e6-4e63-b481-1b397d5231d4
6252023-11-20T07:51:04.037ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_a09c9ef2-b44c-46b9-8f18-372e76dd8249") failed: Property not found. retry in 768.377026ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_a09c9ef2-b44c-46b9-8f18-372e76dd8249
6262023-11-20T07:51:04.193ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_da996e81-243f-4d6a-9a19-416683cfc273") failed: Property not found. retry in 949.682154ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_da996e81-243f-4d6a-9a19-416683cfc273
6272023-11-20T07:51:04.611ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_33c7324a-3441-45ea-8d79-212b82858480") failed: Property not found. retry in 1.03338422s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_33c7324a-3441-45ea-8d79-212b82858480
6282023-11-20T07:51:04.646ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::f858:c2ff:fe1d:5103": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
6292023-11-20T07:51:04.731ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_f561426b-04e6-4e63-b481-1b397d5231d4") failed: Property not found. retry in 829.250963ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_f561426b-04e6-4e63-b481-1b397d5231d4
6302023-11-20T07:51:04.815ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_a09c9ef2-b44c-46b9-8f18-372e76dd8249") failed: Property not found. retry in 1.353966828s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_a09c9ef2-b44c-46b9-8f18-372e76dd8249
6312023-11-20T07:51:05.152ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_da996e81-243f-4d6a-9a19-416683cfc273") failed: Property not found. retry in 1.269536298s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_da996e81-243f-4d6a-9a19-416683cfc273
6322023-11-20T07:51:05.214ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_a3323bbb-65d3-472d-a293-6c510636cffb") failed: Property not found. retry in 1.170305465s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_a3323bbb-65d3-472d-a293-6c510636cffb
6332023-11-20T07:51:05.570ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_f561426b-04e6-4e63-b481-1b397d5231d4") failed: Property not found. retry in 776.208584ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_f561426b-04e6-4e63-b481-1b397d5231d4
6342023-11-20T07:51:05.653ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_33c7324a-3441-45ea-8d79-212b82858480") failed: Property not found. retry in 1.067358878s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_33c7324a-3441-45ea-8d79-212b82858480
6352023-11-20T07:51:06.180ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_a09c9ef2-b44c-46b9-8f18-372e76dd8249") failed: Property not found. retry in 926.151806ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_a09c9ef2-b44c-46b9-8f18-372e76dd8249
6362023-11-20T07:51:06.356ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_f561426b-04e6-4e63-b481-1b397d5231d4") failed: Property not found. retry in 923.054058ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_f561426b-04e6-4e63-b481-1b397d5231d4
6372023-11-20T07:51:06.394ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_a3323bbb-65d3-472d-a293-6c510636cffb") failed: Property not found. retry in 749.013922ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_a3323bbb-65d3-472d-a293-6c510636cffb
6382023-11-20T07:51:06.430ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_da996e81-243f-4d6a-9a19-416683cfc273") failed: Property not found. retry in 848.261228ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_da996e81-243f-4d6a-9a19-416683cfc273
6392023-11-20T07:51:06.731ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_33c7324a-3441-45ea-8d79-212b82858480") failed: Property not found. retry in 1.265066084s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_33c7324a-3441-45ea-8d79-212b82858480
6402023-11-20T07:51:07.116ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_a09c9ef2-b44c-46b9-8f18-372e76dd8249") failed: Property not found. retry in 1.175348188s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_a09c9ef2-b44c-46b9-8f18-372e76dd8249
6412023-11-20T07:51:07.153ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_a3323bbb-65d3-472d-a293-6c510636cffb") failed: Property not found. retry in 1.191033055s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_a3323bbb-65d3-472d-a293-6c510636cffb
6422023-11-20T07:51:07.287ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_da996e81-243f-4d6a-9a19-416683cfc273") failed: Property not found. retry in 1.00012853s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_da996e81-243f-4d6a-9a19-416683cfc273
6432023-11-20T07:51:07.295ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_f561426b-04e6-4e63-b481-1b397d5231d4") failed: Property not found. retry in 819.704416ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_f561426b-04e6-4e63-b481-1b397d5231d4
6442023-11-20T07:51:08.007ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_33c7324a-3441-45ea-8d79-212b82858480") failed: Property not found. retry in 1.425894268s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_33c7324a-3441-45ea-8d79-212b82858480
6452023-11-20T07:51:08.124ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_f561426b-04e6-4e63-b481-1b397d5231d4") failed: Property not found. retry in 543.790923ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_f561426b-04e6-4e63-b481-1b397d5231d4
6462023-11-20T07:51:08.298ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_da996e81-243f-4d6a-9a19-416683cfc273") failed: Property not found. retry in 1.353957756s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_da996e81-243f-4d6a-9a19-416683cfc273
6472023-11-20T07:51:08.307ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_a09c9ef2-b44c-46b9-8f18-372e76dd8249") failed: Property not found. retry in 1.216099975s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_a09c9ef2-b44c-46b9-8f18-372e76dd8249
6482023-11-20T07:51:08.353ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_a3323bbb-65d3-472d-a293-6c510636cffb") failed: Property not found. retry in 1.267819072s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_a3323bbb-65d3-472d-a293-6c510636cffb
6492023-11-20T07:51:08.679ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_f561426b-04e6-4e63-b481-1b397d5231d4") failed: Property not found. retry in 1.446372118s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_f561426b-04e6-4e63-b481-1b397d5231d4
6502023-11-20T07:51:08.939ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:769 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:47954
6512023-11-20T07:51:08.940ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 574 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:47954 req_id = b0aa8d73-2d0e-4286-9ee3-e4fc90d4132d response_code = 200 uri = /baseboard
6522023-11-20T07:51:09.444ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_33c7324a-3441-45ea-8d79-212b82858480") failed: Property not found. retry in 842.351156ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_33c7324a-3441-45ea-8d79-212b82858480
6532023-11-20T07:51:09.534ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_a09c9ef2-b44c-46b9-8f18-372e76dd8249") failed: Property not found. retry in 822.982372ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_a09c9ef2-b44c-46b9-8f18-372e76dd8249
6542023-11-20T07:51:09.630ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_a3323bbb-65d3-472d-a293-6c510636cffb") failed: Property not found. retry in 536.362857ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_a3323bbb-65d3-472d-a293-6c510636cffb
6552023-11-20T07:51:09.648ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::f858:c2ff:fe1d:5103": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
6562023-11-20T07:51:09.662ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_da996e81-243f-4d6a-9a19-416683cfc273") failed: Property not found. retry in 912.274405ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_da996e81-243f-4d6a-9a19-416683cfc273
6572023-11-20T07:51:10.135ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_f561426b-04e6-4e63-b481-1b397d5231d4") failed: Property not found. retry in 1.058674556s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_f561426b-04e6-4e63-b481-1b397d5231d4
6582023-11-20T07:51:10.176ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_a3323bbb-65d3-472d-a293-6c510636cffb") failed: Property not found. retry in 884.20556ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_a3323bbb-65d3-472d-a293-6c510636cffb
6592023-11-20T07:51:10.296ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_33c7324a-3441-45ea-8d79-212b82858480") failed: Property not found. retry in 1.433965178s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_33c7324a-3441-45ea-8d79-212b82858480
6602023-11-20T07:51:10.366ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_a09c9ef2-b44c-46b9-8f18-372e76dd8249") failed: Property not found. retry in 1.172346384s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_a09c9ef2-b44c-46b9-8f18-372e76dd8249
6612023-11-20T07:51:10.583ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_da996e81-243f-4d6a-9a19-416683cfc273") failed: Property not found. retry in 873.150709ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_da996e81-243f-4d6a-9a19-416683cfc273
6622023-11-20T07:51:11.752ZINFOSledAgent (ServiceManager): Writing ledger to /pool/int/a462a7f7-b628-40fe-80ff-4e4189e2d62b/config/.services.json.tmp file = common/src/ledger.rs:197
6632023-11-20T07:51:11.753ZINFOSledAgent (ServiceManager): Writing ledger to /pool/int/b462a7f7-b628-40fe-80ff-4e4189e2d62b/config/.services.json.tmp file = common/src/ledger.rs:197
6642023-11-20T07:51:11.753ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 14354926 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::1]:38094 req_id = 40320188-70a4-4d0f-a0f0-1df624ed55b5 response_code = 204 uri = /services
6652023-11-20T07:51:11.788ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:769 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:47809
6662023-11-20T07:51:11.819ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = Failed to get address: Failed to get address for name oxControlService6/omicron6 in oxz_cockroachdb_33c7324a-3441-45ea-8d79-212b82858480: Zone execution error: Command [/usr/sbin/zlogin oxz_cockroachdb_33c7324a-3441-45ea-8d79-212b82858480 /usr/sbin/ipadm show-addr -p -o ADDR oxControlService6/omicron6] executed and failed with status: exit status: 1 stdout: \n stderr: ipadm: Address object not found file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:841 latency_us = 30876 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::1]:47809 req_id = 7a764f87-b74c-4ed8-9bb9-0d9757b9244d response_code = 500 uri = /cockroachdb
6672023-11-20T07:51:11.819ZWARNSledAgent (RSS): Failed to initialize CockroachDB error = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "7a764f87-b74c-4ed8-9bb9-0d9757b9244d", "content-length": "124", "date": "Mon, 20 Nov 2023 07:51:11 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "7a764f87-b74c-4ed8-9bb9-0d9757b9244d" } file = sled-agent/src/rack_setup/service.rs:737 retry_after = 123.963288ms
6682023-11-20T07:51:11.973ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = Failed to get address: Failed to get address for name oxControlService6/omicron6 in oxz_cockroachdb_33c7324a-3441-45ea-8d79-212b82858480: Zone execution error: Command [/usr/sbin/zlogin oxz_cockroachdb_33c7324a-3441-45ea-8d79-212b82858480 /usr/sbin/ipadm show-addr -p -o ADDR oxControlService6/omicron6] executed and failed with status: exit status: 1 stdout: \n stderr: ipadm: Address object not found file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:841 latency_us = 28403 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::1]:47809 req_id = c2243fc6-c0d2-4d83-ba15-b6ab951876f6 response_code = 500 uri = /cockroachdb
6692023-11-20T07:51:11.973ZWARNSledAgent (RSS): Failed to initialize CockroachDB error = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "c2243fc6-c0d2-4d83-ba15-b6ab951876f6", "content-length": "124", "date": "Mon, 20 Nov 2023 07:51:11 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "c2243fc6-c0d2-4d83-ba15-b6ab951876f6" } file = sled-agent/src/rack_setup/service.rs:737 retry_after = 155.00074ms
6702023-11-20T07:51:12.158ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = Failed to get address: Failed to get address for name oxControlService6/omicron6 in oxz_cockroachdb_33c7324a-3441-45ea-8d79-212b82858480: Zone execution error: Command [/usr/sbin/zlogin oxz_cockroachdb_33c7324a-3441-45ea-8d79-212b82858480 /usr/sbin/ipadm show-addr -p -o ADDR oxControlService6/omicron6] executed and failed with status: exit status: 1 stdout: \n stderr: ipadm: Address object not found file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:841 latency_us = 28392 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::1]:47809 req_id = d01ddf17-8f03-442c-81f6-353d6c85ff88 response_code = 500 uri = /cockroachdb
6712023-11-20T07:51:12.158ZWARNSledAgent (RSS): Failed to initialize CockroachDB error = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "d01ddf17-8f03-442c-81f6-353d6c85ff88", "content-length": "124", "date": "Mon, 20 Nov 2023 07:51:11 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "d01ddf17-8f03-442c-81f6-353d6c85ff88" } file = sled-agent/src/rack_setup/service.rs:737 retry_after = 133.280864ms
6722023-11-20T07:51:12.318ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = Failed to get address: Failed to get address for name oxControlService6/omicron6 in oxz_cockroachdb_33c7324a-3441-45ea-8d79-212b82858480: Zone execution error: Command [/usr/sbin/zlogin oxz_cockroachdb_33c7324a-3441-45ea-8d79-212b82858480 /usr/sbin/ipadm show-addr -p -o ADDR oxControlService6/omicron6] executed and failed with status: exit status: 1 stdout: \n stderr: ipadm: Address object not found file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:841 latency_us = 25780 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::1]:47809 req_id = 5b70ed58-7cab-4585-8842-e668c63c1c85 response_code = 500 uri = /cockroachdb
6732023-11-20T07:51:12.318ZWARNSledAgent (RSS): Failed to initialize CockroachDB error = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "5b70ed58-7cab-4585-8842-e668c63c1c85", "content-length": "124", "date": "Mon, 20 Nov 2023 07:51:11 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "5b70ed58-7cab-4585-8842-e668c63c1c85" } file = sled-agent/src/rack_setup/service.rs:737 retry_after = 164.577617ms
6742023-11-20T07:51:12.509ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = Failed to get address: Failed to get address for name oxControlService6/omicron6 in oxz_cockroachdb_33c7324a-3441-45ea-8d79-212b82858480: Zone execution error: Command [/usr/sbin/zlogin oxz_cockroachdb_33c7324a-3441-45ea-8d79-212b82858480 /usr/sbin/ipadm show-addr -p -o ADDR oxControlService6/omicron6] executed and failed with status: exit status: 1 stdout: \n stderr: ipadm: Address object not found file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:841 latency_us = 26105 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::1]:47809 req_id = b9a22310-78ba-4493-aa36-b4ebba5f98a7 response_code = 500 uri = /cockroachdb
6752023-11-20T07:51:12.510ZWARNSledAgent (RSS): Failed to initialize CockroachDB error = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "b9a22310-78ba-4493-aa36-b4ebba5f98a7", "content-length": "124", "date": "Mon, 20 Nov 2023 07:51:11 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "b9a22310-78ba-4493-aa36-b4ebba5f98a7" } file = sled-agent/src/rack_setup/service.rs:737 retry_after = 220.831773ms
6762023-11-20T07:51:12.759ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = Failed to get address: Failed to get address for name oxControlService6/omicron6 in oxz_cockroachdb_33c7324a-3441-45ea-8d79-212b82858480: Zone execution error: Command [/usr/sbin/zlogin oxz_cockroachdb_33c7324a-3441-45ea-8d79-212b82858480 /usr/sbin/ipadm show-addr -p -o ADDR oxControlService6/omicron6] executed and failed with status: exit status: 1 stdout: \n stderr: ipadm: Address object not found file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:841 latency_us = 26529 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::1]:47809 req_id = 38c1eede-1a5d-45a7-a315-fd3e7937696d response_code = 500 uri = /cockroachdb
6772023-11-20T07:51:12.759ZWARNSledAgent (RSS): Failed to initialize CockroachDB error = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "38c1eede-1a5d-45a7-a315-fd3e7937696d", "content-length": "124", "date": "Mon, 20 Nov 2023 07:51:11 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "38c1eede-1a5d-45a7-a315-fd3e7937696d" } file = sled-agent/src/rack_setup/service.rs:737 retry_after = 131.4389ms
6782023-11-20T07:51:12.919ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = Failed to get address: Failed to get address for name oxControlService6/omicron6 in oxz_cockroachdb_33c7324a-3441-45ea-8d79-212b82858480: Zone execution error: Command [/usr/sbin/zlogin oxz_cockroachdb_33c7324a-3441-45ea-8d79-212b82858480 /usr/sbin/ipadm show-addr -p -o ADDR oxControlService6/omicron6] executed and failed with status: exit status: 1 stdout: \n stderr: ipadm: Address object not found file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:841 latency_us = 27575 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::1]:47809 req_id = 555d7986-1e3e-4019-8d14-efdcf6f235a3 response_code = 500 uri = /cockroachdb
6792023-11-20T07:51:12.919ZWARNSledAgent (RSS): Failed to initialize CockroachDB error = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "555d7986-1e3e-4019-8d14-efdcf6f235a3", "content-length": "124", "date": "Mon, 20 Nov 2023 07:51:12 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "555d7986-1e3e-4019-8d14-efdcf6f235a3" } file = sled-agent/src/rack_setup/service.rs:737 retry_after = 419.946933ms
6802023-11-20T07:51:13.367ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = Failed to get address: Failed to get address for name oxControlService6/omicron6 in oxz_cockroachdb_33c7324a-3441-45ea-8d79-212b82858480: Zone execution error: Command [/usr/sbin/zlogin oxz_cockroachdb_33c7324a-3441-45ea-8d79-212b82858480 /usr/sbin/ipadm show-addr -p -o ADDR oxControlService6/omicron6] executed and failed with status: exit status: 1 stdout: \n stderr: ipadm: Address object not found file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:841 latency_us = 27013 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::1]:47809 req_id = c5132d78-4ebd-430e-85d9-7c182df6002a response_code = 500 uri = /cockroachdb
6812023-11-20T07:51:13.368ZWARNSledAgent (RSS): Failed to initialize CockroachDB error = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "c5132d78-4ebd-430e-85d9-7c182df6002a", "content-length": "124", "date": "Mon, 20 Nov 2023 07:51:12 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "c5132d78-4ebd-430e-85d9-7c182df6002a" } file = sled-agent/src/rack_setup/service.rs:737 retry_after = 318.186749ms
6822023-11-20T07:51:13.717ZINFOSledAgent (ServiceManager): Initializing CRDB Cluster - sending request to [fd00:1122:3344:101::4]:32221 file = sled-agent/src/services.rs:2336
6832023-11-20T07:51:25.494ZINFOSledAgent (ServiceManager): Formatting CRDB file = sled-agent/src/services.rs:2354
6842023-11-20T07:51:37.889ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = Failed to initialize CockroachDb: Error running command in zone 'oxz_cockroachdb_33c7324a-3441-45ea-8d79-212b82858480': Command [/opt/oxide/cockroachdb/bin/cockroach sql --insecure --host [fd00:1122:3344:101::4]:32221 --file /opt/oxide/cockroachdb/sql/dbinit.sql] executed and failed with status: exit status: 1 stdout: BEGIN\nCREATE DATABASE\nCREATE ROLE\nALTER DEFAULT PRIVILEGES\nCONFIGURE ZONE 1\nCREATE TABLE\nCREATE TABLE\nCREATE INDEX\nCREATE TYPE\nCREATE TABLE\nCREATE INDEX\nCREATE TABLE\nCREATE INDEX\nCREATE TYPE\nCREATE TABLE\nCREATE INDEX\nCREATE INDEX\nCREATE TYPE\nCREATE TABLE\nCREATE INDEX\nCREATE INDEX\nCREATE TABLE\nCREATE INDEX\nCREATE INDEX\n stderr: ERROR: restart transaction: TransactionRetryWithProtoRefreshError: TransactionAbortedError(ABORT_REASON_CLIENT_REJECT): "sql txn" meta={id=863fb5d1 key=/NamespaceTable/30/1/106/0/"public"/4/1 pri=0.01265677 epo=0 ts=1700466693.209694151,2 min=1700466687.205103080,0 seq=190} lock=true stat=PENDING rts=1700466687.205103080,0 wto=false gul=1700466687.705103080,0\nSQLSTATE: 40001\nHINT: See: https://www.cockroachlabs.com/docs/v22.1/transaction-retry-error-reference.html#abort_reason_client_reject\nERROR: restart transaction: TransactionRetryWithProtoRefreshError: TransactionAbortedError(ABORT_REASON_CLIENT_REJECT): "sql txn" meta={id=863fb5d1 key=/NamespaceTable/30/1/106/0/"public"/4/1 pri=0.01265677 epo=0 ts=1700466693.209694151,2 min=1700466687.205103080,0 seq=190} lock=true stat=PENDING rts=1700466687.205103080,0 wto=false gul=1700466687.705103080,0\nSQLSTATE: 40001\nHINT: See: https://www.cockroachlabs.com/docs/v22.1/transaction-retry-error-reference.html#abort_reason_client_reject\nFailed running "sql"\n file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:841 latency_us = 24198603 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::1]:47809 req_id = 3c96ec9c-c5b9-4fa6-9e7b-7b3350f86411 response_code = 500 uri = /cockroachdb
6852023-11-20T07:51:37.890ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::f858:c2ff:fe1d:5103": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
6862023-11-20T07:51:37.891ZWARNSledAgent (RSS): Failed to initialize CockroachDB error = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "3c96ec9c-c5b9-4fa6-9e7b-7b3350f86411", "content-length": "124", "date": "Mon, 20 Nov 2023 07:51:37 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "3c96ec9c-c5b9-4fa6-9e7b-7b3350f86411" } file = sled-agent/src/rack_setup/service.rs:737 retry_after = 629.182397ms
6872023-11-20T07:51:38.556ZINFOSledAgent (ServiceManager): Initializing CRDB Cluster - sending request to [fd00:1122:3344:101::4]:32221 file = sled-agent/src/services.rs:2336
6882023-11-20T07:51:38.677ZINFOSledAgent (ServiceManager): Formatting CRDB file = sled-agent/src/services.rs:2354
6892023-11-20T07:51:41.331ZINFOSledAgent (ServiceManager): Formatting CRDB - Completed file = sled-agent/src/services.rs:2375
6902023-11-20T07:51:41.331ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 2809375 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::1]:47809 req_id = 0b6d9679-2d89-4d84-9d18-f53d104f1e5a response_code = 204 uri = /cockroachdb
6912023-11-20T07:51:41.331ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:769 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:61597
6922023-11-20T07:51:41.331ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 100 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:61597 req_id = 47bcd301-5b75-450f-8387-225cb44585ed response_code = 200 uri = /baseboard
6932023-11-20T07:51:41.365ZINFOSledAgent (RSS): sending service requests... file = sled-agent/src/rack_setup/service.rs:284
6942023-11-20T07:51:41.365ZINFOSledAgent (RSS): initializing sled services: [ServiceZoneRequest { addresses: [fd00:1122:3344:1::1], dataset: Some(DatasetRequest { id: c9a9027a-c2d6-437c-b4f4-de4f53331cd0, name: DatasetName { kind: InternalDns, pool_name: ZpoolName("oxp_14b4dc87-ab46-49fb-a4b4-d361ae214c03") }, service_address: "[fd00:1122:3344:1::1]:5353" }), id: c9a9027a-c2d6-437c-b4f4-de4f53331cd0, services: [ServiceZoneService { details: InternalDns { dns_address: "[fd00:1122:3344:1::1]:53", gz_address: fd00:1122:3344:1::2, gz_address_index: 0, http_address: "[fd00:1122:3344:1::1]:5353" }, id: c9a9027a-c2d6-437c-b4f4-de4f53331cd0 }], zone_type: InternalDns }, ServiceZoneRequest { addresses: [fd00:1122:3344:2::1], dataset: Some(DatasetRequest { id: defece7e-1f41-48b6-a56f-7f3b68d5a311, name: DatasetName { kind: InternalDns, pool_name: ZpoolName("oxp_24b4dc87-ab46-49fb-a4b4-d361ae214c03") }, service_address: "[fd00:1122:3344:2::1]:5353" }), id: defece7e-1f41-48b6-a56f-7f3b68d5a311, services: [ServiceZoneService { details: InternalDns { dns_address: "[fd00:1122:3344:2::1]:53", gz_address: fd00:1122:3344:2::2, gz_address_index: 1, http_address: "[fd00:1122:3344:2::1]:5353" }, id: defece7e-1f41-48b6-a56f-7f3b68d5a311 }], zone_type: InternalDns }, ServiceZoneRequest { addresses: [fd00:1122:3344:3::1], dataset: Some(DatasetRequest { id: 306c7298-6fe3-4323-b3ea-7a7f9075ced3, name: DatasetName { kind: InternalDns, pool_name: ZpoolName("oxp_31bd71cd-4736-4a12-a387-9b74b050396f") }, service_address: "[fd00:1122:3344:3::1]:5353" }), id: 306c7298-6fe3-4323-b3ea-7a7f9075ced3, services: [ServiceZoneService { details: InternalDns { dns_address: "[fd00:1122:3344:3::1]:53", gz_address: fd00:1122:3344:3::2, gz_address_index: 2, http_address: "[fd00:1122:3344:3::1]:5353" }, id: 306c7298-6fe3-4323-b3ea-7a7f9075ced3 }], zone_type: InternalDns }, ServiceZoneRequest { addresses: [fd00:1122:3344:101::3], dataset: Some(DatasetRequest { id: a3323bbb-65d3-472d-a293-6c510636cffb, name: DatasetName { kind: CockroachDb, pool_name: ZpoolName("oxp_14b4dc87-ab46-49fb-a4b4-d361ae214c03") }, service_address: "[fd00:1122:3344:101::3]:32221" }), id: a3323bbb-65d3-472d-a293-6c510636cffb, services: [ServiceZoneService { details: CockroachDb { address: "[fd00:1122:3344:101::3]:32221" }, id: a3323bbb-65d3-472d-a293-6c510636cffb }], zone_type: CockroachDb }, ServiceZoneRequest { addresses: [fd00:1122:3344:101::4], dataset: Some(DatasetRequest { id: 33c7324a-3441-45ea-8d79-212b82858480, name: DatasetName { kind: CockroachDb, pool_name: ZpoolName("oxp_24b4dc87-ab46-49fb-a4b4-d361ae214c03") }, service_address: "[fd00:1122:3344:101::4]:32221" }), id: 33c7324a-3441-45ea-8d79-212b82858480, services: [ServiceZoneService { details: CockroachDb { address: "[fd00:1122:3344:101::4]:32221" }, id: 33c7324a-3441-45ea-8d79-212b82858480 }], zone_type: CockroachDb }, ServiceZoneRequest { addresses: [fd00:1122:3344:101::5], dataset: Some(DatasetRequest { id: f561426b-04e6-4e63-b481-1b397d5231d4, name: DatasetName { kind: CockroachDb, pool_name: ZpoolName("oxp_31bd71cd-4736-4a12-a387-9b74b050396f") }, service_address: "[fd00:1122:3344:101::5]:32221" }), id: f561426b-04e6-4e63-b481-1b397d5231d4, services: [ServiceZoneService { details: CockroachDb { address: "[fd00:1122:3344:101::5]:32221" }, id: f561426b-04e6-4e63-b481-1b397d5231d4 }], zone_type: CockroachDb }, ServiceZoneRequest { addresses: [fd00:1122:3344:101::6], dataset: Some(DatasetRequest { id: da996e81-243f-4d6a-9a19-416683cfc273, name: DatasetName { kind: CockroachDb, pool_name: ZpoolName("oxp_616b26df-e62a-4c68-b506-f4a923d8aaf7") }, service_address: "[fd00:1122:3344:101::6]:32221" }), id: da996e81-243f-4d6a-9a19-416683cfc273, services: [ServiceZoneService { details: CockroachDb { address: "[fd00:1122:3344:101::6]:32221" }, id: da996e81-243f-4d6a-9a19-416683cfc273 }], zone_type: CockroachDb }, ServiceZoneRequest { addresses: [fd00:1122:3344:101::7], dataset: Some(DatasetRequest { id: a09c9ef2-b44c-46b9-8f18-372e76dd8249, name: DatasetName { kind: CockroachDb, pool_name: ZpoolName("oxp_cd70d7f6-2354-4bf2-8012-55bf9eaf7930") }, service_address: "[fd00:1122:3344:101::7]:32221" }), id: a09c9ef2-b44c-46b9-8f18-372e76dd8249, services: [ServiceZoneService { details: CockroachDb { address: "[fd00:1122:3344:101::7]:32221" }, id: a09c9ef2-b44c-46b9-8f18-372e76dd8249 }], zone_type: CockroachDb }, ServiceZoneRequest { addresses: [fd00:1122:3344:101::8], dataset: Some(DatasetRequest { id: 84037cc2-9a57-4b92-aa79-1df7e38b4b09, name: DatasetName { kind: ExternalDns, pool_name: ZpoolName("oxp_14b4dc87-ab46-49fb-a4b4-d361ae214c03") }, service_address: "[fd00:1122:3344:101::8]:5353" }), id: 84037cc2-9a57-4b92-aa79-1df7e38b4b09, services: [ServiceZoneService { details: ExternalDns { dns_address: "10.151.2.164:53", http_address: "[fd00:1122:3344:101::8]:5353", nic: NetworkInterface { id: c01ef95f-95de-4126-8d67-460994581d78, ip: 172.30.1.5, kind: Service(84037cc2-9a57-4b92-aa79-1df7e38b4b09), mac: MacAddr("A8:40:25:FF:82:CB"), name: Name("external-dns-84037cc2-9a57-4b92-aa79-1df7e38b4b09"), primary: true, slot: 0, subnet: V4(Ipv4Net("172.30.1.0/24")), vni: Vni(100) } }, id: 84037cc2-9a57-4b92-aa79-1df7e38b4b09 }], zone_type: ExternalDns }, ServiceZoneRequest { addresses: [fd00:1122:3344:101::9], dataset: Some(DatasetRequest { id: 5dda65cf-da71-4547-b51e-e1cf5d91d287, name: DatasetName { kind: ExternalDns, pool_name: ZpoolName("oxp_24b4dc87-ab46-49fb-a4b4-d361ae214c03") }, service_address: "[fd00:1122:3344:101::9]:5353" }), id: 5dda65cf-da71-4547-b51e-e1cf5d91d287, services: [ServiceZoneService { details: ExternalDns { dns_address: "10.151.2.165:53", http_address: "[fd00:1122:3344:101::9]:5353", nic: NetworkInterface { id: cc29b875-5948-412e-af7a-ce1c0c8e86ae, ip: 172.30.1.6, kind: Service(5dda65cf-da71-4547-b51e-e1cf5d91d287), mac: MacAddr("A8:40:25:FF:F8:6D"), name: Name("external-dns-5dda65cf-da71-4547-b51e-e1cf5d91d287"), primary: true, slot: 0, subnet: V4(Ipv4Net("172.30.1.0/24")), vni: Vni(100) } }, id: 5dda65cf-da71-4547-b51e-e1cf5d91d287 }], zone_type: ExternalDns }, ServiceZoneRequest { addresses: [fd00:1122:3344:101::a], dataset: None, id: 8a5fa6e3-3703-4684-a6e8-b41d55048601, services: [ServiceZoneService { details: Nexus { external_dns_servers: [1.1.1.1, 9.9.9.9], external_ip: 10.151.2.166, external_tls: true, internal_address: "[fd00:1122:3344:101::a]:12221", nic: NetworkInterface { id: ceca0063-034f-4eaf-b17a-618175b29716, ip: 172.30.2.5, kind: Service(8a5fa6e3-3703-4684-a6e8-b41d55048601), mac: MacAddr("A8:40:25:FF:8A:46"), name: Name("nexus-8a5fa6e3-3703-4684-a6e8-b41d55048601"), primary: true, slot: 0, subnet: V4(Ipv4Net("172.30.2.0/24")), vni: Vni(100) } }, id: 8a5fa6e3-3703-4684-a6e8-b41d55048601 }], zone_type: Nexus }, ServiceZoneRequest { addresses: [fd00:1122:3344:101::b], dataset: None, id: dc808808-c29f-4e50-89dc-eb71dee374d3, services: [ServiceZoneService { details: Nexus { external_dns_servers: [1.1.1.1, 9.9.9.9], external_ip: 10.151.2.167, external_tls: true, internal_address: "[fd00:1122:3344:101::b]:12221", nic: NetworkInterface { id: 96cd7458-ba2f-4142-af49-fe66c0c7b8d4, ip: 172.30.2.6, kind: Service(dc808808-c29f-4e50-89dc-eb71dee374d3), mac: MacAddr("A8:40:25:FF:AA:99"), name: Name("nexus-dc808808-c29f-4e50-89dc-eb71dee374d3"), primary: true, slot: 0, subnet: V4(Ipv4Net("172.30.2.0/24")), vni: Vni(100) } }, id: dc808808-c29f-4e50-89dc-eb71dee374d3 }], zone_type: Nexus }, ServiceZoneRequest { addresses: [fd00:1122:3344:101::c], dataset: None, id: 381f1240-83d8-4fb0-9bf7-1e162996eca3, services: [ServiceZoneService { details: Nexus { external_dns_servers: [1.1.1.1, 9.9.9.9], external_ip: 10.151.2.168, external_tls: true, internal_address: "[fd00:1122:3344:101::c]:12221", nic: NetworkInterface { id: bb3f68ea-6a70-4785-aaee-607c1a5127cd, ip: 172.30.2.7, kind: Service(381f1240-83d8-4fb0-9bf7-1e162996eca3), mac: MacAddr("A8:40:25:FF:E0:8A"), name: Name("nexus-381f1240-83d8-4fb0-9bf7-1e162996eca3"), primary: true, slot: 0, subnet: V4(Ipv4Net("172.30.2.0/24")), vni: Vni(100) } }, id: 381f1240-83d8-4fb0-9bf7-1e162996eca3 }], zone_type: Nexus }, ServiceZoneRequest { addresses: [fd00:1122:3344:101::d], dataset: None, id: 8cb95d51-90ee-4878-942d-5d274a6755b3, services: [ServiceZoneService { details: Oximeter { address: "[fd00:1122:3344:101::d]:12223" }, id: 8cb95d51-90ee-4878-942d-5d274a6755b3 }], zone_type: Oximeter }, ServiceZoneRequest { addresses: [fd00:1122:3344:101::e], dataset: Some(DatasetRequest { id: 847cd99c-7101-48eb-bf5f-29a882b481fe, name: DatasetName { kind: Clickhouse, pool_name: ZpoolName("oxp_14b4dc87-ab46-49fb-a4b4-d361ae214c03") }, service_address: "[fd00:1122:3344:101::e]:8123" }), id: 847cd99c-7101-48eb-bf5f-29a882b481fe, services: [ServiceZoneService { details: Clickhouse { address: "[fd00:1122:3344:101::e]:8123" }, id: 847cd99c-7101-48eb-bf5f-29a882b481fe }], zone_type: Clickhouse }, ServiceZoneRequest { addresses: [fd00:1122:3344:101::f], dataset: None, id: 088f787e-f34a-4c96-b5de-12e3e6780164, services: [ServiceZoneService { details: CruciblePantry { address: "[fd00:1122:3344:101::f]:17000" }, id: 088f787e-f34a-4c96-b5de-12e3e6780164 }], zone_type: CruciblePantry }, ServiceZoneRequest { addresses: [fd00:1122:3344:101::10], dataset: None, id: 16b5abb9-d189-4e4e-96c0-108193957b3d, services: [ServiceZoneService { details: CruciblePantry { address: "[fd00:1122:3344:101::10]:17000" }, id: 16b5abb9-d189-4e4e-96c0-108193957b3d }], zone_type: CruciblePantry }, ServiceZoneRequest { addresses: [fd00:1122:3344:101::11], dataset: None, id: f17d1012-45a4-4e32-a4cd-f647739a92e6, services: [ServiceZoneService { details: CruciblePantry { address: "[fd00:1122:3344:101::11]:17000" }, id: f17d1012-45a4-4e32-a4cd-f647739a92e6 }], zone_type: CruciblePantry }, ServiceZoneRequest { addresses: [fd00:1122:3344:101::12], dataset: Some(DatasetRequest { id: a9743596-df71-421e-b6dc-00d0636ce520, name: DatasetName { kind: Crucible, pool_name: ZpoolName("oxp_14b4dc87-ab46-49fb-a4b4-d361ae214c03") }, service_address: "[fd00:1122:3344:101::12]:32345" }), id: a9743596-df71-421e-b6dc-00d0636ce520, services: [ServiceZoneService { details: Crucible { address: "[fd00:1122:3344:101::12]:32345" }, id: a9743596-df71-421e-b6dc-00d0636ce520 }], zone_type: Crucible }, ServiceZoneRequest { addresses: [fd00:1122:3344:101::13], dataset: Some(DatasetRequest { id: 07e03e9a-beed-4e41-b732-48a7946b5a69, name: DatasetName { kind: Crucible, pool_name: ZpoolName("oxp_24b4dc87-ab46-49fb-a4b4-d361ae214c03") }, service_address: "[fd00:1122:3344:101::13]:32345" }), id: 07e03e9a-beed-4e41-b732-48a7946b5a69, services: [ServiceZoneService { details: Crucible { address: "[fd00:1122:3344:101::13]:32345" }, id: 07e03e9a-beed-4e41-b732-48a7946b5a69 }], zone_type: Crucible }, ServiceZoneRequest { addresses: [fd00:1122:3344:101::14], dataset: Some(DatasetRequest { id: f8c559b1-5e66-4e93-9478-2feb1a69060f, name: DatasetName { kind: Crucible, pool_name: ZpoolName("oxp_31bd71cd-4736-4a12-a387-9b74b050396f") }, service_address: "[fd00:1122:3344:101::14]:32345" }), id: f8c559b1-5e66-4e93-9478-2feb1a69060f, services: [ServiceZoneService { details: Crucible { address: "[fd00:1122:3344:101::14]:32345" }, id: f8c559b1-5e66-4e93-9478-2feb1a69060f }], zone_type: Crucible }, ServiceZoneRequest { addresses: [fd00:1122:3344:101::15], dataset: Some(DatasetRequest { id: 7ac38bb8-58b9-46d6-877a-58d68deadc2c, name: DatasetName { kind: Crucible, pool_name: ZpoolName("oxp_616b26df-e62a-4c68-b506-f4a923d8aaf7") }, service_address: "[fd00:1122:3344:101::15]:32345" }), id: 7ac38bb8-58b9-46d6-877a-58d68deadc2c, services: [ServiceZoneService { details: Crucible { address: "[fd00:1122:3344:101::15]:32345" }, id: 7ac38bb8-58b9-46d6-877a-58d68deadc2c }], zone_type: Crucible }, ServiceZoneRequest { addresses: [fd00:1122:3344:101::16], dataset: Some(DatasetRequest { id: 249188b9-13b4-4193-9568-84db6fc17529, name: DatasetName { kind: Crucible, pool_name: ZpoolName("oxp_cd70d7f6-2354-4bf2-8012-55bf9eaf7930") }, service_address: "[fd00:1122:3344:101::16]:32345" }), id: 249188b9-13b4-4193-9568-84db6fc17529, services: [ServiceZoneService { details: Crucible { address: "[fd00:1122:3344:101::16]:32345" }, id: 249188b9-13b4-4193-9568-84db6fc17529 }], zone_type: Crucible }, ServiceZoneRequest { addresses: [fd00:1122:3344:101::17], dataset: Some(DatasetRequest { id: 05afc584-303a-494e-958d-48a2486cccde, name: DatasetName { kind: Crucible, pool_name: ZpoolName("oxp_ceb4461c-cf56-4719-ad3c-14430bfdfb60") }, service_address: "[fd00:1122:3344:101::17]:32345" }), id: 05afc584-303a-494e-958d-48a2486cccde, services: [ServiceZoneService { details: Crucible { address: "[fd00:1122:3344:101::17]:32345" }, id: 05afc584-303a-494e-958d-48a2486cccde }], zone_type: Crucible }, ServiceZoneRequest { addresses: [fd00:1122:3344:101::18], dataset: Some(DatasetRequest { id: 201086a7-56f2-4de0-8aee-210e5cfefed7, name: DatasetName { kind: Crucible, pool_name: ZpoolName("oxp_d462a7f7-b628-40fe-80ff-4e4189e2d62b") }, service_address: "[fd00:1122:3344:101::18]:32345" }), id: 201086a7-56f2-4de0-8aee-210e5cfefed7, services: [ServiceZoneService { details: Crucible { address: "[fd00:1122:3344:101::18]:32345" }, id: 201086a7-56f2-4de0-8aee-210e5cfefed7 }], zone_type: Crucible }, ServiceZoneRequest { addresses: [fd00:1122:3344:101::19], dataset: Some(DatasetRequest { id: 3abd98fd-69cc-4f10-9866-db07b595b88e, name: DatasetName { kind: Crucible, pool_name: ZpoolName("oxp_e4b4dc87-ab46-49fb-a4b4-d361ae214c03") }, service_address: "[fd00:1122:3344:101::19]:32345" }), id: 3abd98fd-69cc-4f10-9866-db07b595b88e, services: [ServiceZoneService { details: Crucible { address: "[fd00:1122:3344:101::19]:32345" }, id: 3abd98fd-69cc-4f10-9866-db07b595b88e }], zone_type: Crucible }, ServiceZoneRequest { addresses: [fd00:1122:3344:101::1a], dataset: Some(DatasetRequest { id: 8bbe15f8-a976-42cd-ae90-1c7c4c5d3fea, name: DatasetName { kind: Crucible, pool_name: ZpoolName("oxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03") }, service_address: "[fd00:1122:3344:101::1a]:32345" }), id: 8bbe15f8-a976-42cd-ae90-1c7c4c5d3fea, services: [ServiceZoneService { details: Crucible { address: "[fd00:1122:3344:101::1a]:32345" }, id: 8bbe15f8-a976-42cd-ae90-1c7c4c5d3fea }], zone_type: Crucible }, ServiceZoneRequest { addresses: [fd00:1122:3344:101::1b], dataset: None, id: 505fd4f2-4a1e-4353-87c4-9b9b02e962ea, services: [ServiceZoneService { details: BoundaryNtp { address: "[fd00:1122:3344:101::1b]:123", dns_servers: [1.1.1.1, 9.9.9.9], domain: None, nic: NetworkInterface { id: 04c6421f-f7d5-4dd8-8ec2-99fd604b460c, ip: 172.30.3.5, kind: Service(505fd4f2-4a1e-4353-87c4-9b9b02e962ea), mac: MacAddr("A8:40:25:FF:E1:30"), name: Name("ntp-505fd4f2-4a1e-4353-87c4-9b9b02e962ea"), primary: true, slot: 0, subnet: V4(Ipv4Net("172.30.3.0/24")), vni: Vni(100) }, ntp_servers: ["0.pool.ntp.org"], snat_cfg: SourceNatConfig { first_port: 0, ip: 10.151.2.169, last_port: 16383 } }, id: 505fd4f2-4a1e-4353-87c4-9b9b02e962ea }], zone_type: Ntp }] file = sled-agent/src/rack_setup/service.rs:286
6952023-11-20T07:51:41.366ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:769 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:65090
6962023-11-20T07:51:41.367ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: c9a9027a-c2d6-437c-b4f4-de4f53331cd0, dataset_name: DatasetName { pool_name: ZpoolName { id: 14b4dc87-ab46-49fb-a4b4-d361ae214c03, kind: External }, kind: InternalDns }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } }) file = sled-storage/src/manager.rs:346
6972023-11-20T07:51:41.367ZINFOSledAgent (StorageManager): add_dataset: NewFilesystemRequest { dataset_id: c9a9027a-c2d6-437c-b4f4-de4f53331cd0, dataset_name: DatasetName { pool_name: ZpoolName { id: 14b4dc87-ab46-49fb-a4b4-d361ae214c03, kind: External }, kind: InternalDns }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } } file = sled-storage/src/manager.rs:564
6982023-11-20T07:51:41.391ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: defece7e-1f41-48b6-a56f-7f3b68d5a311, dataset_name: DatasetName { pool_name: ZpoolName { id: 24b4dc87-ab46-49fb-a4b4-d361ae214c03, kind: External }, kind: InternalDns }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } }) file = sled-storage/src/manager.rs:346
6992023-11-20T07:51:41.393ZINFOSledAgent (StorageManager): add_dataset: NewFilesystemRequest { dataset_id: defece7e-1f41-48b6-a56f-7f3b68d5a311, dataset_name: DatasetName { pool_name: ZpoolName { id: 24b4dc87-ab46-49fb-a4b4-d361ae214c03, kind: External }, kind: InternalDns }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } } file = sled-storage/src/manager.rs:564
7002023-11-20T07:51:41.411ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 306c7298-6fe3-4323-b3ea-7a7f9075ced3, dataset_name: DatasetName { pool_name: ZpoolName { id: 31bd71cd-4736-4a12-a387-9b74b050396f, kind: External }, kind: InternalDns }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } }) file = sled-storage/src/manager.rs:346
7012023-11-20T07:51:41.413ZINFOSledAgent (StorageManager): add_dataset: NewFilesystemRequest { dataset_id: 306c7298-6fe3-4323-b3ea-7a7f9075ced3, dataset_name: DatasetName { pool_name: ZpoolName { id: 31bd71cd-4736-4a12-a387-9b74b050396f, kind: External }, kind: InternalDns }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } } file = sled-storage/src/manager.rs:564
7022023-11-20T07:51:41.430ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: a3323bbb-65d3-472d-a293-6c510636cffb, dataset_name: DatasetName { pool_name: ZpoolName { id: 14b4dc87-ab46-49fb-a4b4-d361ae214c03, kind: External }, kind: CockroachDb }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } }) file = sled-storage/src/manager.rs:346
7032023-11-20T07:51:41.430ZINFOSledAgent (StorageManager): add_dataset: NewFilesystemRequest { dataset_id: a3323bbb-65d3-472d-a293-6c510636cffb, dataset_name: DatasetName { pool_name: ZpoolName { id: 14b4dc87-ab46-49fb-a4b4-d361ae214c03, kind: External }, kind: CockroachDb }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } } file = sled-storage/src/manager.rs:564
7042023-11-20T07:51:41.449ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 33c7324a-3441-45ea-8d79-212b82858480, dataset_name: DatasetName { pool_name: ZpoolName { id: 24b4dc87-ab46-49fb-a4b4-d361ae214c03, kind: External }, kind: CockroachDb }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } }) file = sled-storage/src/manager.rs:346
7052023-11-20T07:51:41.449ZINFOSledAgent (StorageManager): add_dataset: NewFilesystemRequest { dataset_id: 33c7324a-3441-45ea-8d79-212b82858480, dataset_name: DatasetName { pool_name: ZpoolName { id: 24b4dc87-ab46-49fb-a4b4-d361ae214c03, kind: External }, kind: CockroachDb }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } } file = sled-storage/src/manager.rs:564
7062023-11-20T07:51:41.467ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: f561426b-04e6-4e63-b481-1b397d5231d4, dataset_name: DatasetName { pool_name: ZpoolName { id: 31bd71cd-4736-4a12-a387-9b74b050396f, kind: External }, kind: CockroachDb }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } }) file = sled-storage/src/manager.rs:346
7072023-11-20T07:51:41.467ZINFOSledAgent (StorageManager): add_dataset: NewFilesystemRequest { dataset_id: f561426b-04e6-4e63-b481-1b397d5231d4, dataset_name: DatasetName { pool_name: ZpoolName { id: 31bd71cd-4736-4a12-a387-9b74b050396f, kind: External }, kind: CockroachDb }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } } file = sled-storage/src/manager.rs:564
7082023-11-20T07:51:41.486ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: da996e81-243f-4d6a-9a19-416683cfc273, dataset_name: DatasetName { pool_name: ZpoolName { id: 616b26df-e62a-4c68-b506-f4a923d8aaf7, kind: External }, kind: CockroachDb }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } }) file = sled-storage/src/manager.rs:346
7092023-11-20T07:51:41.488ZINFOSledAgent (StorageManager): add_dataset: NewFilesystemRequest { dataset_id: da996e81-243f-4d6a-9a19-416683cfc273, dataset_name: DatasetName { pool_name: ZpoolName { id: 616b26df-e62a-4c68-b506-f4a923d8aaf7, kind: External }, kind: CockroachDb }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } } file = sled-storage/src/manager.rs:564
7102023-11-20T07:51:41.503ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: a09c9ef2-b44c-46b9-8f18-372e76dd8249, dataset_name: DatasetName { pool_name: ZpoolName { id: cd70d7f6-2354-4bf2-8012-55bf9eaf7930, kind: External }, kind: CockroachDb }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } }) file = sled-storage/src/manager.rs:346
7112023-11-20T07:51:41.503ZINFOSledAgent (StorageManager): add_dataset: NewFilesystemRequest { dataset_id: a09c9ef2-b44c-46b9-8f18-372e76dd8249, dataset_name: DatasetName { pool_name: ZpoolName { id: cd70d7f6-2354-4bf2-8012-55bf9eaf7930, kind: External }, kind: CockroachDb }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } } file = sled-storage/src/manager.rs:564
7122023-11-20T07:51:41.522ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 84037cc2-9a57-4b92-aa79-1df7e38b4b09, dataset_name: DatasetName { pool_name: ZpoolName { id: 14b4dc87-ab46-49fb-a4b4-d361ae214c03, kind: External }, kind: ExternalDns }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } }) file = sled-storage/src/manager.rs:346
7132023-11-20T07:51:41.522ZINFOSledAgent (StorageManager): add_dataset: NewFilesystemRequest { dataset_id: 84037cc2-9a57-4b92-aa79-1df7e38b4b09, dataset_name: DatasetName { pool_name: ZpoolName { id: 14b4dc87-ab46-49fb-a4b4-d361ae214c03, kind: External }, kind: ExternalDns }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } } file = sled-storage/src/manager.rs:564
7142023-11-20T07:51:41.600ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 5dda65cf-da71-4547-b51e-e1cf5d91d287, dataset_name: DatasetName { pool_name: ZpoolName { id: 24b4dc87-ab46-49fb-a4b4-d361ae214c03, kind: External }, kind: ExternalDns }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } }) file = sled-storage/src/manager.rs:346
7152023-11-20T07:51:41.600ZINFOSledAgent (StorageManager): add_dataset: NewFilesystemRequest { dataset_id: 5dda65cf-da71-4547-b51e-e1cf5d91d287, dataset_name: DatasetName { pool_name: ZpoolName { id: 24b4dc87-ab46-49fb-a4b4-d361ae214c03, kind: External }, kind: ExternalDns }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } } file = sled-storage/src/manager.rs:564
7162023-11-20T07:51:41.685ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 847cd99c-7101-48eb-bf5f-29a882b481fe, dataset_name: DatasetName { pool_name: ZpoolName { id: 14b4dc87-ab46-49fb-a4b4-d361ae214c03, kind: External }, kind: Clickhouse }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } }) file = sled-storage/src/manager.rs:346
7172023-11-20T07:51:41.685ZINFOSledAgent (StorageManager): add_dataset: NewFilesystemRequest { dataset_id: 847cd99c-7101-48eb-bf5f-29a882b481fe, dataset_name: DatasetName { pool_name: ZpoolName { id: 14b4dc87-ab46-49fb-a4b4-d361ae214c03, kind: External }, kind: Clickhouse }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } } file = sled-storage/src/manager.rs:564
7182023-11-20T07:51:41.766ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: a9743596-df71-421e-b6dc-00d0636ce520, dataset_name: DatasetName { pool_name: ZpoolName { id: 14b4dc87-ab46-49fb-a4b4-d361ae214c03, kind: External }, kind: Crucible }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } }) file = sled-storage/src/manager.rs:346
7192023-11-20T07:51:41.766ZINFOSledAgent (StorageManager): add_dataset: NewFilesystemRequest { dataset_id: a9743596-df71-421e-b6dc-00d0636ce520, dataset_name: DatasetName { pool_name: ZpoolName { id: 14b4dc87-ab46-49fb-a4b4-d361ae214c03, kind: External }, kind: Crucible }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } } file = sled-storage/src/manager.rs:564
7202023-11-20T07:51:41.839ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 07e03e9a-beed-4e41-b732-48a7946b5a69, dataset_name: DatasetName { pool_name: ZpoolName { id: 24b4dc87-ab46-49fb-a4b4-d361ae214c03, kind: External }, kind: Crucible }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } }) file = sled-storage/src/manager.rs:346
7212023-11-20T07:51:41.839ZINFOSledAgent (StorageManager): add_dataset: NewFilesystemRequest { dataset_id: 07e03e9a-beed-4e41-b732-48a7946b5a69, dataset_name: DatasetName { pool_name: ZpoolName { id: 24b4dc87-ab46-49fb-a4b4-d361ae214c03, kind: External }, kind: Crucible }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } } file = sled-storage/src/manager.rs:564
7222023-11-20T07:51:41.907ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: f8c559b1-5e66-4e93-9478-2feb1a69060f, dataset_name: DatasetName { pool_name: ZpoolName { id: 31bd71cd-4736-4a12-a387-9b74b050396f, kind: External }, kind: Crucible }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } }) file = sled-storage/src/manager.rs:346
7232023-11-20T07:51:41.907ZINFOSledAgent (StorageManager): add_dataset: NewFilesystemRequest { dataset_id: f8c559b1-5e66-4e93-9478-2feb1a69060f, dataset_name: DatasetName { pool_name: ZpoolName { id: 31bd71cd-4736-4a12-a387-9b74b050396f, kind: External }, kind: Crucible }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } } file = sled-storage/src/manager.rs:564
7242023-11-20T07:51:41.981ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 7ac38bb8-58b9-46d6-877a-58d68deadc2c, dataset_name: DatasetName { pool_name: ZpoolName { id: 616b26df-e62a-4c68-b506-f4a923d8aaf7, kind: External }, kind: Crucible }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } }) file = sled-storage/src/manager.rs:346
7252023-11-20T07:51:41.981ZINFOSledAgent (StorageManager): add_dataset: NewFilesystemRequest { dataset_id: 7ac38bb8-58b9-46d6-877a-58d68deadc2c, dataset_name: DatasetName { pool_name: ZpoolName { id: 616b26df-e62a-4c68-b506-f4a923d8aaf7, kind: External }, kind: Crucible }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } } file = sled-storage/src/manager.rs:564
7262023-11-20T07:51:42.055ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 249188b9-13b4-4193-9568-84db6fc17529, dataset_name: DatasetName { pool_name: ZpoolName { id: cd70d7f6-2354-4bf2-8012-55bf9eaf7930, kind: External }, kind: Crucible }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } }) file = sled-storage/src/manager.rs:346
7272023-11-20T07:51:42.055ZINFOSledAgent (StorageManager): add_dataset: NewFilesystemRequest { dataset_id: 249188b9-13b4-4193-9568-84db6fc17529, dataset_name: DatasetName { pool_name: ZpoolName { id: cd70d7f6-2354-4bf2-8012-55bf9eaf7930, kind: External }, kind: Crucible }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } } file = sled-storage/src/manager.rs:564
7282023-11-20T07:51:42.130ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 05afc584-303a-494e-958d-48a2486cccde, dataset_name: DatasetName { pool_name: ZpoolName { id: ceb4461c-cf56-4719-ad3c-14430bfdfb60, kind: External }, kind: Crucible }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } }) file = sled-storage/src/manager.rs:346
7292023-11-20T07:51:42.130ZINFOSledAgent (StorageManager): add_dataset: NewFilesystemRequest { dataset_id: 05afc584-303a-494e-958d-48a2486cccde, dataset_name: DatasetName { pool_name: ZpoolName { id: ceb4461c-cf56-4719-ad3c-14430bfdfb60, kind: External }, kind: Crucible }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } } file = sled-storage/src/manager.rs:564
7302023-11-20T07:51:42.213ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 201086a7-56f2-4de0-8aee-210e5cfefed7, dataset_name: DatasetName { pool_name: ZpoolName { id: d462a7f7-b628-40fe-80ff-4e4189e2d62b, kind: External }, kind: Crucible }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } }) file = sled-storage/src/manager.rs:346
7312023-11-20T07:51:42.213ZINFOSledAgent (StorageManager): add_dataset: NewFilesystemRequest { dataset_id: 201086a7-56f2-4de0-8aee-210e5cfefed7, dataset_name: DatasetName { pool_name: ZpoolName { id: d462a7f7-b628-40fe-80ff-4e4189e2d62b, kind: External }, kind: Crucible }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } } file = sled-storage/src/manager.rs:564
7322023-11-20T07:51:42.286ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 3abd98fd-69cc-4f10-9866-db07b595b88e, dataset_name: DatasetName { pool_name: ZpoolName { id: e4b4dc87-ab46-49fb-a4b4-d361ae214c03, kind: External }, kind: Crucible }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } }) file = sled-storage/src/manager.rs:346
7332023-11-20T07:51:42.286ZINFOSledAgent (StorageManager): add_dataset: NewFilesystemRequest { dataset_id: 3abd98fd-69cc-4f10-9866-db07b595b88e, dataset_name: DatasetName { pool_name: ZpoolName { id: e4b4dc87-ab46-49fb-a4b4-d361ae214c03, kind: External }, kind: Crucible }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } } file = sled-storage/src/manager.rs:564
7342023-11-20T07:51:42.361ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 8bbe15f8-a976-42cd-ae90-1c7c4c5d3fea, dataset_name: DatasetName { pool_name: ZpoolName { id: f4b4dc87-ab46-49fb-a4b4-d361ae214c03, kind: External }, kind: Crucible }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } }) file = sled-storage/src/manager.rs:346
7352023-11-20T07:51:42.362ZINFOSledAgent (StorageManager): add_dataset: NewFilesystemRequest { dataset_id: 8bbe15f8-a976-42cd-ae90-1c7c4c5d3fea, dataset_name: DatasetName { pool_name: ZpoolName { id: f4b4dc87-ab46-49fb-a4b4-d361ae214c03, kind: External }, kind: Crucible }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } } file = sled-storage/src/manager.rs:564
7362023-11-20T07:51:42.431ZINFOSledAgent (StorageManager): Received GetLatestResources(Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) }) file = sled-storage/src/manager.rs:346
7372023-11-20T07:51:42.431ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/a462a7f7-b628-40fe-80ff-4e4189e2d62b/config/services.json file = common/src/ledger.rs:175
7382023-11-20T07:51:42.431ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/b462a7f7-b628-40fe-80ff-4e4189e2d62b/config/services.json file = common/src/ledger.rs:175
7392023-11-20T07:51:42.431ZINFOSledAgent (StorageManager): Received GetLatestResources(Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) }) file = sled-storage/src/manager.rs:346
7402023-11-20T07:51:42.437ZINFOSledAgent (ServiceManager): Resolving switch zone addresses in DNS file = sled-agent/src/bootstrap/early_networking.rs:253
7412023-11-20T07:51:42.437ZINFOSledAgent (StorageManager): Received GetLatestResources(Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) }) file = sled-storage/src/manager.rs:346
7422023-11-20T07:51:42.437ZINFOSledAgent (StorageManager): Received GetLatestResources(Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) }) file = sled-storage/src/manager.rs:346
7432023-11-20T07:51:42.439ZINFOSledAgent (ServiceManager): Resolving switch zone addresses in DNS file = sled-agent/src/bootstrap/early_networking.rs:253
7442023-11-20T07:51:42.446ZINFOSledAgent (ServiceManager): Resolving switch zone addresses in DNS file = sled-agent/src/bootstrap/early_networking.rs:253
7452023-11-20T07:51:42.446ZINFOSledAgent (StorageManager): Received GetLatestResources(Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) }) file = sled-storage/src/manager.rs:346
7462023-11-20T07:51:42.446ZINFOSledAgent (StorageManager): Received GetLatestResources(Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) }) file = sled-storage/src/manager.rs:346
7472023-11-20T07:51:42.446ZINFOSledAgent (StorageManager): Received GetLatestResources(Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) }) file = sled-storage/src/manager.rs:346
7482023-11-20T07:51:42.446ZINFOSledAgent (StorageManager): Received GetLatestResources(Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) }) file = sled-storage/src/manager.rs:346
7492023-11-20T07:51:42.446ZINFOSledAgent (StorageManager): Received GetLatestResources(Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) }) file = sled-storage/src/manager.rs:346
7502023-11-20T07:51:42.451ZINFOSledAgent (ServiceManager): Resolving switch zone addresses in DNS file = sled-agent/src/bootstrap/early_networking.rs:253
7512023-11-20T07:51:42.452ZINFOSledAgent (StorageManager): Received GetLatestResources(Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) }) file = sled-storage/src/manager.rs:346
7522023-11-20T07:51:42.452ZINFOSledAgent (StorageManager): Received GetLatestResources(Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) }) file = sled-storage/src/manager.rs:346
7532023-11-20T07:51:42.452ZINFOSledAgent (StorageManager): Received GetLatestResources(Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) }) file = sled-storage/src/manager.rs:346
7542023-11-20T07:51:42.453ZINFOSledAgent (ServiceManager): Resolving switch zone addresses in DNS file = sled-agent/src/bootstrap/early_networking.rs:253
7552023-11-20T07:51:42.458ZINFOSledAgent (StorageManager): Received GetLatestResources(Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) }) file = sled-storage/src/manager.rs:346
7562023-11-20T07:51:42.458ZINFOSledAgent (StorageManager): Received GetLatestResources(Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) }) file = sled-storage/src/manager.rs:346
7572023-11-20T07:51:42.458ZINFOSledAgent (StorageManager): Received GetLatestResources(Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) }) file = sled-storage/src/manager.rs:346
7582023-11-20T07:51:42.458ZINFOSledAgent (StorageManager): Received GetLatestResources(Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) }) file = sled-storage/src/manager.rs:346
7592023-11-20T07:51:42.729ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_a9743596-df71-421e-b6dc-00d0636ce520 file = illumos-utils/src/zone.rs:314
7602023-11-20T07:51:42.733ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_3abd98fd-69cc-4f10-9866-db07b595b88e file = illumos-utils/src/zone.rs:314
7612023-11-20T07:51:42.739ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_pantry_088f787e-f34a-4c96-b5de-12e3e6780164 file = illumos-utils/src/zone.rs:314
7622023-11-20T07:51:42.744ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_pantry_16b5abb9-d189-4e4e-96c0-108193957b3d file = illumos-utils/src/zone.rs:314
7632023-11-20T07:51:42.749ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_clickhouse_847cd99c-7101-48eb-bf5f-29a882b481fe file = illumos-utils/src/zone.rs:314
7642023-11-20T07:51:42.753ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_f8c559b1-5e66-4e93-9478-2feb1a69060f file = illumos-utils/src/zone.rs:314
7652023-11-20T07:51:42.758ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_05afc584-303a-494e-958d-48a2486cccde file = illumos-utils/src/zone.rs:314
7662023-11-20T07:51:42.764ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_7ac38bb8-58b9-46d6-877a-58d68deadc2c file = illumos-utils/src/zone.rs:314
7672023-11-20T07:51:42.769ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_07e03e9a-beed-4e41-b732-48a7946b5a69 file = illumos-utils/src/zone.rs:314
7682023-11-20T07:51:42.773ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_249188b9-13b4-4193-9568-84db6fc17529 file = illumos-utils/src/zone.rs:314
7692023-11-20T07:51:42.778ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_a9743596-df71-421e-b6dc-00d0636ce520 file = illumos-utils/src/zone.rs:353
7702023-11-20T07:51:42.783ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_3abd98fd-69cc-4f10-9866-db07b595b88e file = illumos-utils/src/zone.rs:353
7712023-11-20T07:51:42.788ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_pantry_088f787e-f34a-4c96-b5de-12e3e6780164 file = illumos-utils/src/zone.rs:353
7722023-11-20T07:51:42.793ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_pantry_16b5abb9-d189-4e4e-96c0-108193957b3d file = illumos-utils/src/zone.rs:353
7732023-11-20T07:51:42.798ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_clickhouse_847cd99c-7101-48eb-bf5f-29a882b481fe file = illumos-utils/src/zone.rs:353
7742023-11-20T07:51:42.802ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_f8c559b1-5e66-4e93-9478-2feb1a69060f file = illumos-utils/src/zone.rs:353
7752023-11-20T07:51:42.808ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_05afc584-303a-494e-958d-48a2486cccde file = illumos-utils/src/zone.rs:353
7762023-11-20T07:51:42.813ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_7ac38bb8-58b9-46d6-877a-58d68deadc2c file = illumos-utils/src/zone.rs:353
7772023-11-20T07:51:42.818ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_07e03e9a-beed-4e41-b732-48a7946b5a69 file = illumos-utils/src/zone.rs:353
7782023-11-20T07:51:42.823ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_249188b9-13b4-4193-9568-84db6fc17529 file = illumos-utils/src/zone.rs:353
7792023-11-20T07:51:42.829ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_oximeter_8cb95d51-90ee-4878-942d-5d274a6755b3 file = illumos-utils/src/zone.rs:314
7802023-11-20T07:51:42.835ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_201086a7-56f2-4de0-8aee-210e5cfefed7 file = illumos-utils/src/zone.rs:314
7812023-11-20T07:51:42.841ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_8bbe15f8-a976-42cd-ae90-1c7c4c5d3fea file = illumos-utils/src/zone.rs:314
7822023-11-20T07:51:42.847ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_pantry_f17d1012-45a4-4e32-a4cd-f647739a92e6 file = illumos-utils/src/zone.rs:314
7832023-11-20T07:51:42.855ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_oximeter_8cb95d51-90ee-4878-942d-5d274a6755b3 file = illumos-utils/src/zone.rs:353
7842023-11-20T07:51:42.892ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::f858:c2ff:fe1d:5103": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
7852023-11-20T07:51:42.910ZINFOSledAgent (ServiceManager): Querying MGS to determine switch location addr = fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:270
7862023-11-20T07:51:42.952ZINFOSledAgent (ServiceManager): Querying MGS to determine switch location addr = fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:270
7872023-11-20T07:51:42.952ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_201086a7-56f2-4de0-8aee-210e5cfefed7 file = illumos-utils/src/zone.rs:353
7882023-11-20T07:51:42.958ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_8bbe15f8-a976-42cd-ae90-1c7c4c5d3fea file = illumos-utils/src/zone.rs:353
7892023-11-20T07:51:42.963ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_pantry_f17d1012-45a4-4e32-a4cd-f647739a92e6 file = illumos-utils/src/zone.rs:353
7902023-11-20T07:51:43.004ZINFOSledAgent (ServiceManager): Querying MGS to determine switch location addr = fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:270
7912023-11-20T07:51:43.045ZINFOSledAgent (ServiceManager): Querying MGS to determine switch location addr = fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:270
7922023-11-20T07:51:43.049ZINFOSledAgent (ServiceManager): Found Switch0 at fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:295
7932023-11-20T07:51:43.049ZINFOSledAgent (ServiceManager): Successfully looked up all expected switch zone underlay addresses addrs = {Switch0: fd00:1122:3344:101::2} file = sled-agent/src/bootstrap/early_networking.rs:187
7942023-11-20T07:51:43.118ZINFOSledAgent (PortManager): Created OPTE port file = illumos-utils/src/opte/port_manager.rs:383 port = Port { inner: PortInner { name: "opte1", _ip: 172.30.2.6, mac: MacAddr6([168, 64, 37, 255, 170, 153]), slot: 0, vni: Vni { inner: 100 }, gateway: Gateway { mac: MacAddr6([168, 64, 37, 255, 119, 119]), ip: 172.30.2.1 }, vnic: "vopte1" } }
7952023-11-20T07:51:43.118ZINFOSledAgent (ServiceManager): creating NAT entry for service file = sled-agent/src/services.rs:893 service = ServiceZoneService { id: dc808808-c29f-4e50-89dc-eb71dee374d3, details: Nexus { internal_address: [fd00:1122:3344:101::b]:12221, external_ip: 10.151.2.167, nic: NetworkInterface { id: 96cd7458-ba2f-4142-af49-fe66c0c7b8d4, kind: Service { id: dc808808-c29f-4e50-89dc-eb71dee374d3 }, name: Name("nexus-dc808808-c29f-4e50-89dc-eb71dee374d3"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 170, 153])), subnet: V4(Ipv4Net(Ipv4Network { addr: 172.30.2.0, prefix: 24 })), vni: Vni(100), primary: true, slot: 0 }, external_tls: true, external_dns_servers: [1.1.1.1, 9.9.9.9] } }
7962023-11-20T07:51:43.156ZINFOSledAgent (ServiceManager): Querying MGS to determine switch location addr = fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:270
7972023-11-20T07:51:43.160ZINFOSledAgent (ServiceManager): Found Switch0 at fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:295
7982023-11-20T07:51:43.160ZINFOSledAgent (ServiceManager): Successfully looked up all expected switch zone underlay addresses addrs = {Switch0: fd00:1122:3344:101::2} file = sled-agent/src/bootstrap/early_networking.rs:187
7992023-11-20T07:51:43.215ZINFOSledAgent (PortManager): Created OPTE port file = illumos-utils/src/opte/port_manager.rs:383 port = Port { inner: PortInner { name: "opte2", _ip: 172.30.1.5, mac: MacAddr6([168, 64, 37, 255, 130, 203]), slot: 0, vni: Vni { inner: 100 }, gateway: Gateway { mac: MacAddr6([168, 64, 37, 255, 119, 119]), ip: 172.30.1.1 }, vnic: "vopte2" } }
8002023-11-20T07:51:43.215ZINFOSledAgent (ServiceManager): creating NAT entry for service file = sled-agent/src/services.rs:893 service = ServiceZoneService { id: 84037cc2-9a57-4b92-aa79-1df7e38b4b09, details: ExternalDns { http_address: [fd00:1122:3344:101::8]:5353, dns_address: 10.151.2.164:53, nic: NetworkInterface { id: c01ef95f-95de-4126-8d67-460994581d78, kind: Service { id: 84037cc2-9a57-4b92-aa79-1df7e38b4b09 }, name: Name("external-dns-84037cc2-9a57-4b92-aa79-1df7e38b4b09"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 130, 203])), subnet: V4(Ipv4Net(Ipv4Network { addr: 172.30.1.0, prefix: 24 })), vni: Vni(100), primary: true, slot: 0 } } }
8012023-11-20T07:51:43.217ZINFOSledAgent (ServiceManager): Found Switch0 at fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:295
8022023-11-20T07:51:43.217ZINFOSledAgent (ServiceManager): Successfully looked up all expected switch zone underlay addresses addrs = {Switch0: fd00:1122:3344:101::2} file = sled-agent/src/bootstrap/early_networking.rs:187
8032023-11-20T07:51:43.268ZINFOSledAgent (PortManager): Created OPTE port file = illumos-utils/src/opte/port_manager.rs:383 port = Port { inner: PortInner { name: "opte3", _ip: 172.30.2.7, mac: MacAddr6([168, 64, 37, 255, 224, 138]), slot: 0, vni: Vni { inner: 100 }, gateway: Gateway { mac: MacAddr6([168, 64, 37, 255, 119, 119]), ip: 172.30.2.1 }, vnic: "vopte3" } }
8042023-11-20T07:51:43.268ZINFOSledAgent (ServiceManager): creating NAT entry for service file = sled-agent/src/services.rs:893 service = ServiceZoneService { id: 381f1240-83d8-4fb0-9bf7-1e162996eca3, details: Nexus { internal_address: [fd00:1122:3344:101::c]:12221, external_ip: 10.151.2.168, nic: NetworkInterface { id: bb3f68ea-6a70-4785-aaee-607c1a5127cd, kind: Service { id: 381f1240-83d8-4fb0-9bf7-1e162996eca3 }, name: Name("nexus-381f1240-83d8-4fb0-9bf7-1e162996eca3"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 224, 138])), subnet: V4(Ipv4Net(Ipv4Network { addr: 172.30.2.0, prefix: 24 })), vni: Vni(100), primary: true, slot: 0 }, external_tls: true, external_dns_servers: [1.1.1.1, 9.9.9.9] } }
8052023-11-20T07:51:43.272ZINFOSledAgent (ServiceManager): Found Switch0 at fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:295
8062023-11-20T07:51:43.272ZINFOSledAgent (ServiceManager): Successfully looked up all expected switch zone underlay addresses addrs = {Switch0: fd00:1122:3344:101::2} file = sled-agent/src/bootstrap/early_networking.rs:187
8072023-11-20T07:51:43.332ZINFOSledAgent (PortManager): Created OPTE port file = illumos-utils/src/opte/port_manager.rs:383 port = Port { inner: PortInner { name: "opte4", _ip: 172.30.1.6, mac: MacAddr6([168, 64, 37, 255, 248, 109]), slot: 0, vni: Vni { inner: 100 }, gateway: Gateway { mac: MacAddr6([168, 64, 37, 255, 119, 119]), ip: 172.30.1.1 }, vnic: "vopte4" } }
8082023-11-20T07:51:43.332ZINFOSledAgent (ServiceManager): creating NAT entry for service file = sled-agent/src/services.rs:893 service = ServiceZoneService { id: 5dda65cf-da71-4547-b51e-e1cf5d91d287, details: ExternalDns { http_address: [fd00:1122:3344:101::9]:5353, dns_address: 10.151.2.165:53, nic: NetworkInterface { id: cc29b875-5948-412e-af7a-ce1c0c8e86ae, kind: Service { id: 5dda65cf-da71-4547-b51e-e1cf5d91d287 }, name: Name("external-dns-5dda65cf-da71-4547-b51e-e1cf5d91d287"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 248, 109])), subnet: V4(Ipv4Net(Ipv4Network { addr: 172.30.1.0, prefix: 24 })), vni: Vni(100), primary: true, slot: 0 } } }
8092023-11-20T07:51:43.333ZINFOSledAgent (ServiceManager): no nat entry found for: V4( Ipv4Network { addr: 10.151.2.167, prefix: 32, }, ) file = clients/dpd-client/src/lib.rs:98
8102023-11-20T07:51:43.334ZINFOSledAgent (ServiceManager): creating nat entry for: V4( Ipv4Network { addr: 10.151.2.167, prefix: 32, }, ) file = clients/dpd-client/src/lib.rs:105
8112023-11-20T07:51:43.334ZINFOSledAgent (ServiceManager): no nat entry found for: V4( Ipv4Network { addr: 10.151.2.164, prefix: 32, }, ) file = clients/dpd-client/src/lib.rs:98
8122023-11-20T07:51:43.334ZINFOSledAgent (ServiceManager): creating nat entry for: V4( Ipv4Network { addr: 10.151.2.164, prefix: 32, }, ) file = clients/dpd-client/src/lib.rs:105
8132023-11-20T07:51:43.335ZINFOSledAgent (ServiceManager): no nat entry found for: V4( Ipv4Network { addr: 10.151.2.165, prefix: 32, }, ) file = clients/dpd-client/src/lib.rs:98
8142023-11-20T07:51:43.335ZINFOSledAgent (ServiceManager): creating nat entry for: V4( Ipv4Network { addr: 10.151.2.165, prefix: 32, }, ) file = clients/dpd-client/src/lib.rs:105
8152023-11-20T07:51:43.335ZINFOSledAgent (ServiceManager): creation of nat entry successful for: V4( Ipv4Network { addr: 10.151.2.167, prefix: 32, }, ) file = clients/dpd-client/src/lib.rs:133
8162023-11-20T07:51:43.337ZINFOSledAgent (StorageManager): Received GetLatestResources(Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) }) file = sled-storage/src/manager.rs:346
8172023-11-20T07:51:43.339ZINFOSledAgent (ServiceManager): Found Switch0 at fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:295
8182023-11-20T07:51:43.339ZINFOSledAgent (ServiceManager): Successfully looked up all expected switch zone underlay addresses addrs = {Switch0: fd00:1122:3344:101::2} file = sled-agent/src/bootstrap/early_networking.rs:187
8192023-11-20T07:51:43.390ZINFOSledAgent (PortManager): Created OPTE port file = illumos-utils/src/opte/port_manager.rs:383 port = Port { inner: PortInner { name: "opte5", _ip: 172.30.2.5, mac: MacAddr6([168, 64, 37, 255, 138, 70]), slot: 0, vni: Vni { inner: 100 }, gateway: Gateway { mac: MacAddr6([168, 64, 37, 255, 119, 119]), ip: 172.30.2.1 }, vnic: "vopte5" } }
8202023-11-20T07:51:43.390ZINFOSledAgent (ServiceManager): creating NAT entry for service file = sled-agent/src/services.rs:893 service = ServiceZoneService { id: 8a5fa6e3-3703-4684-a6e8-b41d55048601, details: Nexus { internal_address: [fd00:1122:3344:101::a]:12221, external_ip: 10.151.2.166, nic: NetworkInterface { id: ceca0063-034f-4eaf-b17a-618175b29716, kind: Service { id: 8a5fa6e3-3703-4684-a6e8-b41d55048601 }, name: Name("nexus-8a5fa6e3-3703-4684-a6e8-b41d55048601"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 138, 70])), subnet: V4(Ipv4Net(Ipv4Network { addr: 172.30.2.0, prefix: 24 })), vni: Vni(100), primary: true, slot: 0 }, external_tls: true, external_dns_servers: [1.1.1.1, 9.9.9.9] } }
8212023-11-20T07:51:43.390ZINFOSledAgent (ServiceManager): creation of nat entry successful for: V4( Ipv4Network { addr: 10.151.2.164, prefix: 32, }, ) file = clients/dpd-client/src/lib.rs:133
8222023-11-20T07:51:43.393ZINFOSledAgent (ServiceManager): no nat entry found for: V4( Ipv4Network { addr: 10.151.2.168, prefix: 32, }, ) file = clients/dpd-client/src/lib.rs:98
8232023-11-20T07:51:43.393ZINFOSledAgent (ServiceManager): creating nat entry for: V4( Ipv4Network { addr: 10.151.2.168, prefix: 32, }, ) file = clients/dpd-client/src/lib.rs:105
8242023-11-20T07:51:43.393ZINFOSledAgent (StorageManager): Received GetLatestResources(Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) }) file = sled-storage/src/manager.rs:346
8252023-11-20T07:51:43.393ZINFOSledAgent (ServiceManager): creation of nat entry successful for: V4( Ipv4Network { addr: 10.151.2.165, prefix: 32, }, ) file = clients/dpd-client/src/lib.rs:133
8262023-11-20T07:51:43.425ZINFOSledAgent (StorageManager): Received GetLatestResources(Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) }) file = sled-storage/src/manager.rs:346
8272023-11-20T07:51:43.451ZINFOSledAgent (ServiceManager): creation of nat entry successful for: V4( Ipv4Network { addr: 10.151.2.168, prefix: 32, }, ) file = clients/dpd-client/src/lib.rs:133
8282023-11-20T07:51:43.478ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_nexus_dc808808-c29f-4e50-89dc-eb71dee374d3 file = illumos-utils/src/zone.rs:314
8292023-11-20T07:51:43.484ZINFOSledAgent (StorageManager): Received GetLatestResources(Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) }) file = sled-storage/src/manager.rs:346
8302023-11-20T07:51:43.484ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_external_dns_84037cc2-9a57-4b92-aa79-1df7e38b4b09 file = illumos-utils/src/zone.rs:314
8312023-11-20T07:51:43.519ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_external_dns_5dda65cf-da71-4547-b51e-e1cf5d91d287 file = illumos-utils/src/zone.rs:314
8322023-11-20T07:51:43.525ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_nexus_dc808808-c29f-4e50-89dc-eb71dee374d3 file = illumos-utils/src/zone.rs:353
8332023-11-20T07:51:43.530ZINFOSledAgent (ServiceManager): no nat entry found for: V4( Ipv4Network { addr: 10.151.2.166, prefix: 32, }, ) file = clients/dpd-client/src/lib.rs:98
8342023-11-20T07:51:43.530ZINFOSledAgent (ServiceManager): creating nat entry for: V4( Ipv4Network { addr: 10.151.2.166, prefix: 32, }, ) file = clients/dpd-client/src/lib.rs:105
8352023-11-20T07:51:43.531ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_external_dns_84037cc2-9a57-4b92-aa79-1df7e38b4b09 file = illumos-utils/src/zone.rs:353
8362023-11-20T07:51:43.539ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_nexus_381f1240-83d8-4fb0-9bf7-1e162996eca3 file = illumos-utils/src/zone.rs:314
8372023-11-20T07:51:43.546ZINFOSledAgent (ServiceManager): creation of nat entry successful for: V4( Ipv4Network { addr: 10.151.2.166, prefix: 32, }, ) file = clients/dpd-client/src/lib.rs:133
8382023-11-20T07:51:43.549ZINFOSledAgent (StorageManager): Received GetLatestResources(Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) }) file = sled-storage/src/manager.rs:346
8392023-11-20T07:51:43.549ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_external_dns_5dda65cf-da71-4547-b51e-e1cf5d91d287 file = illumos-utils/src/zone.rs:353
8402023-11-20T07:51:43.590ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_nexus_381f1240-83d8-4fb0-9bf7-1e162996eca3 file = illumos-utils/src/zone.rs:353
8412023-11-20T07:51:43.649ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_nexus_8a5fa6e3-3703-4684-a6e8-b41d55048601 file = illumos-utils/src/zone.rs:314
8422023-11-20T07:51:43.692ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_nexus_8a5fa6e3-3703-4684-a6e8-b41d55048601 file = illumos-utils/src/zone.rs:353
8432023-11-20T07:51:45.267ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_pantry_16b5abb9-d189-4e4e-96c0-108193957b3d: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/crucible/pantry"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value="oxControlService13"/> <propval type="astring" name="gateway" value="fd00:1122:3344:101::1"/> <propval type="astring" name="listen_addr" value="fd00:1122:3344:101::10"/> <propval type="astring" name="listen_port" value="17000"/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
8442023-11-20T07:51:45.267ZINFOSledAgent (ServiceManager): Zone booting file = illumos-utils/src/running_zone.rs:509 zone = oxz_crucible_pantry_16b5abb9-d189-4e4e-96c0-108193957b3d
8452023-11-20T07:51:45.400ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_pantry_f17d1012-45a4-4e32-a4cd-f647739a92e6: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/crucible/pantry"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value="oxControlService23"/> <propval type="astring" name="gateway" value="fd00:1122:3344:101::1"/> <propval type="astring" name="listen_addr" value="fd00:1122:3344:101::11"/> <propval type="astring" name="listen_port" value="17000"/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
8462023-11-20T07:51:45.400ZINFOSledAgent (ServiceManager): Zone booting file = illumos-utils/src/running_zone.rs:509 zone = oxz_crucible_pantry_f17d1012-45a4-4e32-a4cd-f647739a92e6
8472023-11-20T07:51:45.517ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_07e03e9a-beed-4e41-b732-48a7946b5a69: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/crucible/agent"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value="oxControlService18"/> <propval type="astring" name="gateway" value="fd00:1122:3344:101::1"/> <propval type="astring" name="dataset" value="oxp_24b4dc87-ab46-49fb-a4b4-d361ae214c03/crucible"/> <propval type="astring" name="listen_addr" value="fd00:1122:3344:101::13"/> <propval type="astring" name="listen_port" value="32345"/> <propval type="astring" name="uuid" value="77b93e9a-8c22-4c1e-bd31-7923693a2b3f"/> <propval type="astring" name="store" value="/data"/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
8482023-11-20T07:51:45.517ZINFOSledAgent (ServiceManager): Zone booting file = illumos-utils/src/running_zone.rs:509 zone = oxz_crucible_07e03e9a-beed-4e41-b732-48a7946b5a69
8492023-11-20T07:51:45.636ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_249188b9-13b4-4193-9568-84db6fc17529: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/crucible/agent"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value="oxControlService19"/> <propval type="astring" name="gateway" value="fd00:1122:3344:101::1"/> <propval type="astring" name="dataset" value="oxp_cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crucible"/> <propval type="astring" name="listen_addr" value="fd00:1122:3344:101::16"/> <propval type="astring" name="listen_port" value="32345"/> <propval type="astring" name="uuid" value="5dc19e2f-1216-4e1e-8704-cff299ca33ee"/> <propval type="astring" name="store" value="/data"/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
8502023-11-20T07:51:45.636ZINFOSledAgent (ServiceManager): Zone booting file = illumos-utils/src/running_zone.rs:509 zone = oxz_crucible_249188b9-13b4-4193-9568-84db6fc17529
8512023-11-20T07:51:45.666ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_7ac38bb8-58b9-46d6-877a-58d68deadc2c: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/crucible/agent"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value="oxControlService17"/> <propval type="astring" name="gateway" value="fd00:1122:3344:101::1"/> <propval type="astring" name="dataset" value="oxp_616b26df-e62a-4c68-b506-f4a923d8aaf7/crucible"/> <propval type="astring" name="listen_addr" value="fd00:1122:3344:101::15"/> <propval type="astring" name="listen_port" value="32345"/> <propval type="astring" name="uuid" value="91d2b9c2-ab8d-40e0-95ce-06bb90c028ab"/> <propval type="astring" name="store" value="/data"/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
8522023-11-20T07:51:45.666ZINFOSledAgent (ServiceManager): Zone booting file = illumos-utils/src/running_zone.rs:509 zone = oxz_crucible_7ac38bb8-58b9-46d6-877a-58d68deadc2c
8532023-11-20T07:51:45.677ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_a9743596-df71-421e-b6dc-00d0636ce520: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/crucible/agent"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value="oxControlService10"/> <propval type="astring" name="gateway" value="fd00:1122:3344:101::1"/> <propval type="astring" name="dataset" value="oxp_14b4dc87-ab46-49fb-a4b4-d361ae214c03/crucible"/> <propval type="astring" name="listen_addr" value="fd00:1122:3344:101::12"/> <propval type="astring" name="listen_port" value="32345"/> <propval type="astring" name="uuid" value="b83bb2af-68f8-4a37-9f47-790de4b38de4"/> <propval type="astring" name="store" value="/data"/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
8542023-11-20T07:51:45.678ZINFOSledAgent (ServiceManager): Zone booting file = illumos-utils/src/running_zone.rs:509 zone = oxz_crucible_a9743596-df71-421e-b6dc-00d0636ce520
8552023-11-20T07:51:45.707ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_f17d1012-45a4-4e32-a4cd-f647739a92e6") failed: Property not found. retry in 57.62082ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_f17d1012-45a4-4e32-a4cd-f647739a92e6
8562023-11-20T07:51:45.739ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_16b5abb9-d189-4e4e-96c0-108193957b3d") failed: Property not found. retry in 39.334091ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_16b5abb9-d189-4e4e-96c0-108193957b3d
8572023-11-20T07:51:45.750ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_05afc584-303a-494e-958d-48a2486cccde: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/crucible/agent"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value="oxControlService16"/> <propval type="astring" name="gateway" value="fd00:1122:3344:101::1"/> <propval type="astring" name="dataset" value="oxp_ceb4461c-cf56-4719-ad3c-14430bfdfb60/crucible"/> <propval type="astring" name="listen_addr" value="fd00:1122:3344:101::17"/> <propval type="astring" name="listen_port" value="32345"/> <propval type="astring" name="uuid" value="f3a1c46d-bff0-4961-ac43-5c2c23dea14f"/> <propval type="astring" name="store" value="/data"/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
8582023-11-20T07:51:45.750ZINFOSledAgent (ServiceManager): Zone booting file = illumos-utils/src/running_zone.rs:509 zone = oxz_crucible_05afc584-303a-494e-958d-48a2486cccde
8592023-11-20T07:51:45.777ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_f17d1012-45a4-4e32-a4cd-f647739a92e6") failed: Property not found. retry in 57.385246ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_f17d1012-45a4-4e32-a4cd-f647739a92e6
8602023-11-20T07:51:45.816ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_16b5abb9-d189-4e4e-96c0-108193957b3d") failed: Property not found. retry in 116.735666ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_16b5abb9-d189-4e4e-96c0-108193957b3d
8612023-11-20T07:51:45.931ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_07e03e9a-beed-4e41-b732-48a7946b5a69") failed: Property not found. retry in 58.626157ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_07e03e9a-beed-4e41-b732-48a7946b5a69
8622023-11-20T07:51:45.950ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_f17d1012-45a4-4e32-a4cd-f647739a92e6") failed: Property not found. retry in 236.044394ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_f17d1012-45a4-4e32-a4cd-f647739a92e6
8632023-11-20T07:51:45.970ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_16b5abb9-d189-4e4e-96c0-108193957b3d") failed: Property not found. retry in 119.781265ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_16b5abb9-d189-4e4e-96c0-108193957b3d
8642023-11-20T07:51:46.016ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_07e03e9a-beed-4e41-b732-48a7946b5a69") failed: Property not found. retry in 107.592661ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_07e03e9a-beed-4e41-b732-48a7946b5a69
8652023-11-20T07:51:46.123ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_7ac38bb8-58b9-46d6-877a-58d68deadc2c") failed: Property not found. retry in 63.30253ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_7ac38bb8-58b9-46d6-877a-58d68deadc2c
8662023-11-20T07:51:46.149ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_16b5abb9-d189-4e4e-96c0-108193957b3d") failed: Property not found. retry in 414.940724ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_16b5abb9-d189-4e4e-96c0-108193957b3d
8672023-11-20T07:51:46.171ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_07e03e9a-beed-4e41-b732-48a7946b5a69") failed: Property not found. retry in 182.902826ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_07e03e9a-beed-4e41-b732-48a7946b5a69
8682023-11-20T07:51:46.419ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_249188b9-13b4-4193-9568-84db6fc17529") failed: Property not found. retry in 44.127512ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_249188b9-13b4-4193-9568-84db6fc17529
8692023-11-20T07:51:46.562ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_7ac38bb8-58b9-46d6-877a-58d68deadc2c") failed: Property not found. retry in 109.06204ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_7ac38bb8-58b9-46d6-877a-58d68deadc2c
8702023-11-20T07:51:46.647ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_07e03e9a-beed-4e41-b732-48a7946b5a69") failed: Property not found. retry in 451.937034ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_07e03e9a-beed-4e41-b732-48a7946b5a69
8712023-11-20T07:51:46.680ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_249188b9-13b4-4193-9568-84db6fc17529") failed: Property not found. retry in 133.814244ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_249188b9-13b4-4193-9568-84db6fc17529
8722023-11-20T07:51:46.716ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_f17d1012-45a4-4e32-a4cd-f647739a92e6") failed: Property not found. retry in 436.362114ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_f17d1012-45a4-4e32-a4cd-f647739a92e6
8732023-11-20T07:51:46.762ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_16b5abb9-d189-4e4e-96c0-108193957b3d") failed: Property not found. retry in 947.224083ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_16b5abb9-d189-4e4e-96c0-108193957b3d
8742023-11-20T07:51:46.813ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_05afc584-303a-494e-958d-48a2486cccde") failed: Property not found. retry in 27.412659ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_05afc584-303a-494e-958d-48a2486cccde
8752023-11-20T07:51:47.013ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_7ac38bb8-58b9-46d6-877a-58d68deadc2c") failed: Property not found. retry in 119.923088ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_7ac38bb8-58b9-46d6-877a-58d68deadc2c
8762023-11-20T07:51:47.059ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_249188b9-13b4-4193-9568-84db6fc17529") failed: Property not found. retry in 118.912ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_249188b9-13b4-4193-9568-84db6fc17529
8772023-11-20T07:51:47.085ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_05afc584-303a-494e-958d-48a2486cccde") failed: Property not found. retry in 87.236397ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_05afc584-303a-494e-958d-48a2486cccde
8782023-11-20T07:51:47.122ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_07e03e9a-beed-4e41-b732-48a7946b5a69") failed: Property not found. retry in 1.066779319s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_07e03e9a-beed-4e41-b732-48a7946b5a69
8792023-11-20T07:51:47.153ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_7ac38bb8-58b9-46d6-877a-58d68deadc2c") failed: Property not found. retry in 268.933606ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_7ac38bb8-58b9-46d6-877a-58d68deadc2c
8802023-11-20T07:51:47.180ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_f17d1012-45a4-4e32-a4cd-f647739a92e6") failed: Property not found. retry in 544.527905ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_f17d1012-45a4-4e32-a4cd-f647739a92e6
8812023-11-20T07:51:47.205ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_05afc584-303a-494e-958d-48a2486cccde") failed: Property not found. retry in 266.448473ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_05afc584-303a-494e-958d-48a2486cccde
8822023-11-20T07:51:47.223ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_249188b9-13b4-4193-9568-84db6fc17529") failed: Property not found. retry in 429.229159ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_249188b9-13b4-4193-9568-84db6fc17529
8832023-11-20T07:51:47.318ZINFOSledAgent (ServiceManager): Zone booting file = illumos-utils/src/running_zone.rs:509 zone = oxz_external_dns_84037cc2-9a57-4b92-aa79-1df7e38b4b09
8842023-11-20T07:51:47.346ZINFOSledAgent (ServiceManager): Zone booting file = illumos-utils/src/running_zone.rs:509 zone = oxz_external_dns_5dda65cf-da71-4547-b51e-e1cf5d91d287
8852023-11-20T07:51:47.447ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_7ac38bb8-58b9-46d6-877a-58d68deadc2c") failed: Property not found. retry in 1.141883254s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_7ac38bb8-58b9-46d6-877a-58d68deadc2c
8862023-11-20T07:51:47.486ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_05afc584-303a-494e-958d-48a2486cccde") failed: Property not found. retry in 477.198848ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_05afc584-303a-494e-958d-48a2486cccde
8872023-11-20T07:51:47.499ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_pantry_088f787e-f34a-4c96-b5de-12e3e6780164: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/crucible/pantry"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value="oxControlService12"/> <propval type="astring" name="gateway" value="fd00:1122:3344:101::1"/> <propval type="astring" name="listen_addr" value="fd00:1122:3344:101::f"/> <propval type="astring" name="listen_port" value="17000"/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
8882023-11-20T07:51:47.499ZINFOSledAgent (ServiceManager): Zone booting file = illumos-utils/src/running_zone.rs:509 zone = oxz_crucible_pantry_088f787e-f34a-4c96-b5de-12e3e6780164
8892023-11-20T07:51:47.628ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_3abd98fd-69cc-4f10-9866-db07b595b88e: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/crucible/agent"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value="oxControlService11"/> <propval type="astring" name="gateway" value="fd00:1122:3344:101::1"/> <propval type="astring" name="dataset" value="oxp_e4b4dc87-ab46-49fb-a4b4-d361ae214c03/crucible"/> <propval type="astring" name="listen_addr" value="fd00:1122:3344:101::19"/> <propval type="astring" name="listen_port" value="32345"/> <propval type="astring" name="uuid" value="5f877d0d-3c46-4424-b247-2a3249f56f26"/> <propval type="astring" name="store" value="/data"/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
8902023-11-20T07:51:47.629ZINFOSledAgent (ServiceManager): Zone booting file = illumos-utils/src/running_zone.rs:509 zone = oxz_crucible_3abd98fd-69cc-4f10-9866-db07b595b88e
8912023-11-20T07:51:47.669ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_249188b9-13b4-4193-9568-84db6fc17529") failed: Property not found. retry in 923.447762ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_249188b9-13b4-4193-9568-84db6fc17529
8922023-11-20T07:51:47.686ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_84037cc2-9a57-4b92-aa79-1df7e38b4b09") failed: Property not found. retry in 52.151333ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_84037cc2-9a57-4b92-aa79-1df7e38b4b09
8932023-11-20T07:51:47.701ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_5dda65cf-da71-4547-b51e-e1cf5d91d287") failed: Property not found. retry in 48.654433ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_5dda65cf-da71-4547-b51e-e1cf5d91d287
8942023-11-20T07:51:47.724ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_16b5abb9-d189-4e4e-96c0-108193957b3d") failed: Property not found. retry in 1.170313882s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_16b5abb9-d189-4e4e-96c0-108193957b3d
8952023-11-20T07:51:47.738ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_f17d1012-45a4-4e32-a4cd-f647739a92e6") failed: Property not found. retry in 1.366674085s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_f17d1012-45a4-4e32-a4cd-f647739a92e6
8962023-11-20T07:51:47.738ZINFOSledAgent (ServiceManager): Zone booting file = illumos-utils/src/running_zone.rs:509 zone = oxz_oximeter_8cb95d51-90ee-4878-942d-5d274a6755b3
8972023-11-20T07:51:47.753ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_84037cc2-9a57-4b92-aa79-1df7e38b4b09") failed: Property not found. retry in 86.477051ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_84037cc2-9a57-4b92-aa79-1df7e38b4b09
8982023-11-20T07:51:47.763ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_5dda65cf-da71-4547-b51e-e1cf5d91d287") failed: Property not found. retry in 61.42211ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_5dda65cf-da71-4547-b51e-e1cf5d91d287
8992023-11-20T07:51:47.843ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_5dda65cf-da71-4547-b51e-e1cf5d91d287") failed: Property not found. retry in 281.470955ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_5dda65cf-da71-4547-b51e-e1cf5d91d287
9002023-11-20T07:51:47.855ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_84037cc2-9a57-4b92-aa79-1df7e38b4b09") failed: Property not found. retry in 245.21716ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_84037cc2-9a57-4b92-aa79-1df7e38b4b09
9012023-11-20T07:51:47.866ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_088f787e-f34a-4c96-b5de-12e3e6780164") failed: Property not found. retry in 71.969355ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_088f787e-f34a-4c96-b5de-12e3e6780164
9022023-11-20T07:51:47.895ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::f858:c2ff:fe1d:5103": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
9032023-11-20T07:51:47.951ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_088f787e-f34a-4c96-b5de-12e3e6780164") failed: Property not found. retry in 79.296272ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_088f787e-f34a-4c96-b5de-12e3e6780164
9042023-11-20T07:51:47.983ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_05afc584-303a-494e-958d-48a2486cccde") failed: Property not found. retry in 480.575902ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_05afc584-303a-494e-958d-48a2486cccde
9052023-11-20T07:51:48.039ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_3abd98fd-69cc-4f10-9866-db07b595b88e") failed: Property not found. retry in 68.732696ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_3abd98fd-69cc-4f10-9866-db07b595b88e
9062023-11-20T07:51:48.053ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_088f787e-f34a-4c96-b5de-12e3e6780164") failed: Property not found. retry in 295.541243ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_088f787e-f34a-4c96-b5de-12e3e6780164
9072023-11-20T07:51:48.090ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_201086a7-56f2-4de0-8aee-210e5cfefed7: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/crucible/agent"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value="oxControlService21"/> <propval type="astring" name="gateway" value="fd00:1122:3344:101::1"/> <propval type="astring" name="dataset" value="oxp_d462a7f7-b628-40fe-80ff-4e4189e2d62b/crucible"/> <propval type="astring" name="listen_addr" value="fd00:1122:3344:101::18"/> <propval type="astring" name="listen_port" value="32345"/> <propval type="astring" name="uuid" value="47fc41a4-2f5d-4755-bf0f-2a2080f4e8f8"/> <propval type="astring" name="store" value="/data"/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
9082023-11-20T07:51:48.090ZINFOSledAgent (ServiceManager): Zone booting file = illumos-utils/src/running_zone.rs:509 zone = oxz_crucible_201086a7-56f2-4de0-8aee-210e5cfefed7
9092023-11-20T07:51:48.120ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_84037cc2-9a57-4b92-aa79-1df7e38b4b09") failed: Property not found. retry in 293.349272ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_84037cc2-9a57-4b92-aa79-1df7e38b4b09
9102023-11-20T07:51:48.120ZINFOSledAgent (ServiceManager): Zone booting file = illumos-utils/src/running_zone.rs:509 zone = oxz_nexus_dc808808-c29f-4e50-89dc-eb71dee374d3
9112023-11-20T07:51:48.147ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_8cb95d51-90ee-4878-942d-5d274a6755b3") failed: Property not found. retry in 66.796465ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_8cb95d51-90ee-4878-942d-5d274a6755b3
9122023-11-20T07:51:48.160ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_3abd98fd-69cc-4f10-9866-db07b595b88e") failed: Property not found. retry in 105.374384ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_3abd98fd-69cc-4f10-9866-db07b595b88e
9132023-11-20T07:51:48.197ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_5dda65cf-da71-4547-b51e-e1cf5d91d287") failed: Property not found. retry in 513.590578ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_5dda65cf-da71-4547-b51e-e1cf5d91d287
9142023-11-20T07:51:48.218ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_07e03e9a-beed-4e41-b732-48a7946b5a69") failed: Property not found. retry in 1.270295867s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_07e03e9a-beed-4e41-b732-48a7946b5a69
9152023-11-20T07:51:48.218ZINFOSledAgent (ServiceManager): Zone booting file = illumos-utils/src/running_zone.rs:509 zone = oxz_nexus_8a5fa6e3-3703-4684-a6e8-b41d55048601
9162023-11-20T07:51:48.248ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_8cb95d51-90ee-4878-942d-5d274a6755b3") failed: Property not found. retry in 90.050932ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_8cb95d51-90ee-4878-942d-5d274a6755b3
9172023-11-20T07:51:48.294ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_3abd98fd-69cc-4f10-9866-db07b595b88e") failed: Property not found. retry in 111.63295ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_3abd98fd-69cc-4f10-9866-db07b595b88e
9182023-11-20T07:51:48.306ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_8bbe15f8-a976-42cd-ae90-1c7c4c5d3fea: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/crucible/agent"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value="oxControlService22"/> <propval type="astring" name="gateway" value="fd00:1122:3344:101::1"/> <propval type="astring" name="dataset" value="oxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crucible"/> <propval type="astring" name="listen_addr" value="fd00:1122:3344:101::1a"/> <propval type="astring" name="listen_port" value="32345"/> <propval type="astring" name="uuid" value="3fd1f9a2-a678-4e02-b3a1-51b5bb62affd"/> <propval type="astring" name="store" value="/data"/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
9192023-11-20T07:51:48.306ZINFOSledAgent (ServiceManager): Zone booting file = illumos-utils/src/running_zone.rs:509 zone = oxz_crucible_8bbe15f8-a976-42cd-ae90-1c7c4c5d3fea
9202023-11-20T07:51:48.385ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_8cb95d51-90ee-4878-942d-5d274a6755b3") failed: Property not found. retry in 182.876881ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_8cb95d51-90ee-4878-942d-5d274a6755b3
9212023-11-20T07:51:48.404ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_088f787e-f34a-4c96-b5de-12e3e6780164") failed: Property not found. retry in 403.783644ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_088f787e-f34a-4c96-b5de-12e3e6780164
9222023-11-20T07:51:48.420ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_3abd98fd-69cc-4f10-9866-db07b595b88e") failed: Property not found. retry in 508.983569ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_3abd98fd-69cc-4f10-9866-db07b595b88e
9232023-11-20T07:51:48.435ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_84037cc2-9a57-4b92-aa79-1df7e38b4b09") failed: Property not found. retry in 612.512023ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_84037cc2-9a57-4b92-aa79-1df7e38b4b09
9242023-11-20T07:51:48.481ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_05afc584-303a-494e-958d-48a2486cccde") failed: Property not found. retry in 999.601778ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_05afc584-303a-494e-958d-48a2486cccde
9252023-11-20T07:51:48.502ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_201086a7-56f2-4de0-8aee-210e5cfefed7") failed: Property not found. retry in 61.381318ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_201086a7-56f2-4de0-8aee-210e5cfefed7
9262023-11-20T07:51:48.535ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_dc808808-c29f-4e50-89dc-eb71dee374d3") failed: Property not found. retry in 62.742056ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_dc808808-c29f-4e50-89dc-eb71dee374d3
9272023-11-20T07:51:48.576ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_201086a7-56f2-4de0-8aee-210e5cfefed7") failed: Property not found. retry in 72.39847ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_201086a7-56f2-4de0-8aee-210e5cfefed7
9282023-11-20T07:51:48.589ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_8cb95d51-90ee-4878-942d-5d274a6755b3") failed: Property not found. retry in 293.272891ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_8cb95d51-90ee-4878-942d-5d274a6755b3
9292023-11-20T07:51:48.603ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_7ac38bb8-58b9-46d6-877a-58d68deadc2c") failed: Property not found. retry in 678.324472ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_7ac38bb8-58b9-46d6-877a-58d68deadc2c
9302023-11-20T07:51:48.617ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_249188b9-13b4-4193-9568-84db6fc17529") failed: Property not found. retry in 702.301595ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_249188b9-13b4-4193-9568-84db6fc17529
9312023-11-20T07:51:48.627ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_dc808808-c29f-4e50-89dc-eb71dee374d3") failed: Property not found. retry in 146.659583ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_dc808808-c29f-4e50-89dc-eb71dee374d3
9322023-11-20T07:51:48.635ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_8a5fa6e3-3703-4684-a6e8-b41d55048601") failed: Property not found. retry in 25.386552ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_8a5fa6e3-3703-4684-a6e8-b41d55048601
9332023-11-20T07:51:48.636ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_f8c559b1-5e66-4e93-9478-2feb1a69060f: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/crucible/agent"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value="oxControlService15"/> <propval type="astring" name="gateway" value="fd00:1122:3344:101::1"/> <propval type="astring" name="dataset" value="oxp_31bd71cd-4736-4a12-a387-9b74b050396f/crucible"/> <propval type="astring" name="listen_addr" value="fd00:1122:3344:101::14"/> <propval type="astring" name="listen_port" value="32345"/> <propval type="astring" name="uuid" value="984aa43e-85f8-453b-b692-49ff6e90be9c"/> <propval type="astring" name="store" value="/data"/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
9342023-11-20T07:51:48.647ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_8bbe15f8-a976-42cd-ae90-1c7c4c5d3fea") failed: Property not found. retry in 38.614042ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_8bbe15f8-a976-42cd-ae90-1c7c4c5d3fea
9352023-11-20T07:51:48.647ZINFOSledAgent (ServiceManager): Zone booting file = illumos-utils/src/running_zone.rs:509 zone = oxz_crucible_f8c559b1-5e66-4e93-9478-2feb1a69060f
9362023-11-20T07:51:48.664ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_201086a7-56f2-4de0-8aee-210e5cfefed7") failed: Property not found. retry in 280.190798ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_201086a7-56f2-4de0-8aee-210e5cfefed7
9372023-11-20T07:51:48.675ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_8a5fa6e3-3703-4684-a6e8-b41d55048601") failed: Property not found. retry in 92.756585ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_8a5fa6e3-3703-4684-a6e8-b41d55048601
9382023-11-20T07:51:48.697ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_8bbe15f8-a976-42cd-ae90-1c7c4c5d3fea") failed: Property not found. retry in 90.919993ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_8bbe15f8-a976-42cd-ae90-1c7c4c5d3fea
9392023-11-20T07:51:48.723ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_5dda65cf-da71-4547-b51e-e1cf5d91d287") failed: Property not found. retry in 1.109450719s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_5dda65cf-da71-4547-b51e-e1cf5d91d287
9402023-11-20T07:51:48.734ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_a9743596-df71-421e-b6dc-00d0636ce520") failed: Property not found. retry in 45.314634ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_a9743596-df71-421e-b6dc-00d0636ce520
9412023-11-20T07:51:48.779ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_8a5fa6e3-3703-4684-a6e8-b41d55048601") failed: Property not found. retry in 127.961743ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_8a5fa6e3-3703-4684-a6e8-b41d55048601
9422023-11-20T07:51:48.794ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_dc808808-c29f-4e50-89dc-eb71dee374d3") failed: Property not found. retry in 150.026755ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_dc808808-c29f-4e50-89dc-eb71dee374d3
9432023-11-20T07:51:48.803ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_a9743596-df71-421e-b6dc-00d0636ce520") failed: Property not found. retry in 94.275605ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_a9743596-df71-421e-b6dc-00d0636ce520
9442023-11-20T07:51:48.817ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_8bbe15f8-a976-42cd-ae90-1c7c4c5d3fea") failed: Property not found. retry in 299.754772ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_8bbe15f8-a976-42cd-ae90-1c7c4c5d3fea
9452023-11-20T07:51:48.828ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_088f787e-f34a-4c96-b5de-12e3e6780164") failed: Property not found. retry in 1.070805497s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_088f787e-f34a-4c96-b5de-12e3e6780164
9462023-11-20T07:51:48.898ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_8cb95d51-90ee-4878-942d-5d274a6755b3") failed: Property not found. retry in 909.045973ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_8cb95d51-90ee-4878-942d-5d274a6755b3
9472023-11-20T07:51:48.909ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_16b5abb9-d189-4e4e-96c0-108193957b3d") failed: Property not found. retry in 1.454206774s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_16b5abb9-d189-4e4e-96c0-108193957b3d
9482023-11-20T07:51:48.919ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_f8c559b1-5e66-4e93-9478-2feb1a69060f") failed: Property not found. retry in 36.775704ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_f8c559b1-5e66-4e93-9478-2feb1a69060f
9492023-11-20T07:51:48.931ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_a9743596-df71-421e-b6dc-00d0636ce520") failed: Property not found. retry in 198.874108ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_a9743596-df71-421e-b6dc-00d0636ce520
9502023-11-20T07:51:48.943ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_8a5fa6e3-3703-4684-a6e8-b41d55048601") failed: Property not found. retry in 332.580742ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_8a5fa6e3-3703-4684-a6e8-b41d55048601
9512023-11-20T07:51:48.956ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_3abd98fd-69cc-4f10-9866-db07b595b88e") failed: Property not found. retry in 443.161946ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_3abd98fd-69cc-4f10-9866-db07b595b88e
9522023-11-20T07:51:48.966ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_f8c559b1-5e66-4e93-9478-2feb1a69060f") failed: Property not found. retry in 56.650601ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_f8c559b1-5e66-4e93-9478-2feb1a69060f
9532023-11-20T07:51:48.977ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_dc808808-c29f-4e50-89dc-eb71dee374d3") failed: Property not found. retry in 343.136282ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_dc808808-c29f-4e50-89dc-eb71dee374d3
9542023-11-20T07:51:48.994ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_201086a7-56f2-4de0-8aee-210e5cfefed7") failed: Property not found. retry in 318.077088ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_201086a7-56f2-4de0-8aee-210e5cfefed7
9552023-11-20T07:51:49.155ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_f8c559b1-5e66-4e93-9478-2feb1a69060f") failed: Property not found. retry in 258.636574ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_f8c559b1-5e66-4e93-9478-2feb1a69060f
9562023-11-20T07:51:49.344ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_a9743596-df71-421e-b6dc-00d0636ce520") failed: Property not found. retry in 553.4824ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_a9743596-df71-421e-b6dc-00d0636ce520
9572023-11-20T07:51:49.377ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_84037cc2-9a57-4b92-aa79-1df7e38b4b09") failed: Property not found. retry in 708.528462ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_84037cc2-9a57-4b92-aa79-1df7e38b4b09
9582023-11-20T07:51:49.394ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_8bbe15f8-a976-42cd-ae90-1c7c4c5d3fea") failed: Property not found. retry in 241.892096ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_8bbe15f8-a976-42cd-ae90-1c7c4c5d3fea
9592023-11-20T07:51:49.415ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_f17d1012-45a4-4e32-a4cd-f647739a92e6") failed: Property not found. retry in 759.193559ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_f17d1012-45a4-4e32-a4cd-f647739a92e6
9602023-11-20T07:51:49.432ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_3abd98fd-69cc-4f10-9866-db07b595b88e") failed: Property not found. retry in 548.49896ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_3abd98fd-69cc-4f10-9866-db07b595b88e
9612023-11-20T07:51:49.448ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_f8c559b1-5e66-4e93-9478-2feb1a69060f") failed: Property not found. retry in 347.836404ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_f8c559b1-5e66-4e93-9478-2feb1a69060f
9622023-11-20T07:51:49.461ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_7ac38bb8-58b9-46d6-877a-58d68deadc2c") failed: Property not found. retry in 1.492951979s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_7ac38bb8-58b9-46d6-877a-58d68deadc2c
9632023-11-20T07:51:49.474ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_249188b9-13b4-4193-9568-84db6fc17529") failed: Property not found. retry in 1.058586052s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_249188b9-13b4-4193-9568-84db6fc17529
9642023-11-20T07:51:49.488ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_dc808808-c29f-4e50-89dc-eb71dee374d3") failed: Property not found. retry in 585.712356ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_dc808808-c29f-4e50-89dc-eb71dee374d3
9652023-11-20T07:51:49.501ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_8a5fa6e3-3703-4684-a6e8-b41d55048601") failed: Property not found. retry in 475.253725ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_8a5fa6e3-3703-4684-a6e8-b41d55048601
9662023-11-20T07:51:49.515ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_201086a7-56f2-4de0-8aee-210e5cfefed7") failed: Property not found. retry in 464.482715ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_201086a7-56f2-4de0-8aee-210e5cfefed7
9672023-11-20T07:51:49.529ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_05afc584-303a-494e-958d-48a2486cccde") failed: Property not found. retry in 1.160819844s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_05afc584-303a-494e-958d-48a2486cccde
9682023-11-20T07:51:49.542ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_07e03e9a-beed-4e41-b732-48a7946b5a69") failed: Property not found. retry in 635.656094ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_07e03e9a-beed-4e41-b732-48a7946b5a69
9692023-11-20T07:51:49.649ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_8bbe15f8-a976-42cd-ae90-1c7c4c5d3fea") failed: Property not found. retry in 1.170207986s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_8bbe15f8-a976-42cd-ae90-1c7c4c5d3fea
9702023-11-20T07:51:49.810ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_f8c559b1-5e66-4e93-9478-2feb1a69060f") failed: Property not found. retry in 763.991066ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_f8c559b1-5e66-4e93-9478-2feb1a69060f
9712023-11-20T07:51:49.826ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_8cb95d51-90ee-4878-942d-5d274a6755b3") failed: Property not found. retry in 708.806901ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_8cb95d51-90ee-4878-942d-5d274a6755b3
9722023-11-20T07:51:49.886ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_5dda65cf-da71-4547-b51e-e1cf5d91d287") failed: Property not found. retry in 1.176565441s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_5dda65cf-da71-4547-b51e-e1cf5d91d287
9732023-11-20T07:51:49.934ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_a9743596-df71-421e-b6dc-00d0636ce520") failed: Property not found. retry in 689.403581ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_a9743596-df71-421e-b6dc-00d0636ce520
9742023-11-20T07:51:49.976ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_088f787e-f34a-4c96-b5de-12e3e6780164") failed: Property not found. retry in 1.171566711s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_088f787e-f34a-4c96-b5de-12e3e6780164
9752023-11-20T07:51:50.017ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_8a5fa6e3-3703-4684-a6e8-b41d55048601") failed: Property not found. retry in 904.095417ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_8a5fa6e3-3703-4684-a6e8-b41d55048601
9762023-11-20T07:51:50.032ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_3abd98fd-69cc-4f10-9866-db07b595b88e") failed: Property not found. retry in 989.731855ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_3abd98fd-69cc-4f10-9866-db07b595b88e
9772023-11-20T07:51:50.048ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_201086a7-56f2-4de0-8aee-210e5cfefed7") failed: Property not found. retry in 931.893946ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_201086a7-56f2-4de0-8aee-210e5cfefed7
9782023-11-20T07:51:50.090ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_dc808808-c29f-4e50-89dc-eb71dee374d3") failed: Property not found. retry in 821.03985ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_dc808808-c29f-4e50-89dc-eb71dee374d3
9792023-11-20T07:51:50.105ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_84037cc2-9a57-4b92-aa79-1df7e38b4b09") failed: Property not found. retry in 632.500148ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_84037cc2-9a57-4b92-aa79-1df7e38b4b09
9802023-11-20T07:51:50.192ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_f17d1012-45a4-4e32-a4cd-f647739a92e6") failed: Property not found. retry in 1.471818745s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_f17d1012-45a4-4e32-a4cd-f647739a92e6
9812023-11-20T07:51:50.210ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_07e03e9a-beed-4e41-b732-48a7946b5a69") failed: Property not found. retry in 1.396189996s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_07e03e9a-beed-4e41-b732-48a7946b5a69
9822023-11-20T07:51:50.304ZINFOSledAgent (ServiceManager): Zone booting file = illumos-utils/src/running_zone.rs:509 zone = oxz_nexus_381f1240-83d8-4fb0-9bf7-1e162996eca3
9832023-11-20T07:51:50.388ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_16b5abb9-d189-4e4e-96c0-108193957b3d") failed: Property not found. retry in 1.472580313s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_16b5abb9-d189-4e4e-96c0-108193957b3d
9842023-11-20T07:51:50.558ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_249188b9-13b4-4193-9568-84db6fc17529") failed: Property not found. retry in 1.071232793s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_249188b9-13b4-4193-9568-84db6fc17529
9852023-11-20T07:51:50.577ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_8cb95d51-90ee-4878-942d-5d274a6755b3") failed: Property not found. retry in 501.323043ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_8cb95d51-90ee-4878-942d-5d274a6755b3
9862023-11-20T07:51:50.597ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_f8c559b1-5e66-4e93-9478-2feb1a69060f") failed: Property not found. retry in 728.831072ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_f8c559b1-5e66-4e93-9478-2feb1a69060f
9872023-11-20T07:51:50.645ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_a9743596-df71-421e-b6dc-00d0636ce520") failed: Property not found. retry in 943.445813ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_a9743596-df71-421e-b6dc-00d0636ce520
9882023-11-20T07:51:50.658ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_381f1240-83d8-4fb0-9bf7-1e162996eca3") failed: Property not found. retry in 28.355608ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_381f1240-83d8-4fb0-9bf7-1e162996eca3
9892023-11-20T07:51:50.714ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_381f1240-83d8-4fb0-9bf7-1e162996eca3") failed: Property not found. retry in 124.250707ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_381f1240-83d8-4fb0-9bf7-1e162996eca3
9902023-11-20T07:51:50.736ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_05afc584-303a-494e-958d-48a2486cccde") failed: Property not found. retry in 563.362291ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_05afc584-303a-494e-958d-48a2486cccde
9912023-11-20T07:51:50.757ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_84037cc2-9a57-4b92-aa79-1df7e38b4b09") failed: Property not found. retry in 518.960649ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_84037cc2-9a57-4b92-aa79-1df7e38b4b09
9922023-11-20T07:51:50.853ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_8bbe15f8-a976-42cd-ae90-1c7c4c5d3fea") failed: Property not found. retry in 524.495681ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_8bbe15f8-a976-42cd-ae90-1c7c4c5d3fea
9932023-11-20T07:51:50.875ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_381f1240-83d8-4fb0-9bf7-1e162996eca3") failed: Property not found. retry in 276.875583ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_381f1240-83d8-4fb0-9bf7-1e162996eca3
9942023-11-20T07:51:50.933ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_dc808808-c29f-4e50-89dc-eb71dee374d3") failed: Property not found. retry in 875.603541ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_dc808808-c29f-4e50-89dc-eb71dee374d3
9952023-11-20T07:51:50.947ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_8a5fa6e3-3703-4684-a6e8-b41d55048601") failed: Property not found. retry in 1.450357975s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_8a5fa6e3-3703-4684-a6e8-b41d55048601
9962023-11-20T07:51:50.969ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_7ac38bb8-58b9-46d6-877a-58d68deadc2c") failed: Property not found. retry in 1.343983772s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_7ac38bb8-58b9-46d6-877a-58d68deadc2c
9972023-11-20T07:51:51.000ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_201086a7-56f2-4de0-8aee-210e5cfefed7") failed: Property not found. retry in 870.481191ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_201086a7-56f2-4de0-8aee-210e5cfefed7
9982023-11-20T07:51:51.041ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_3abd98fd-69cc-4f10-9866-db07b595b88e") failed: Property not found. retry in 948.697258ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_3abd98fd-69cc-4f10-9866-db07b595b88e
9992023-11-20T07:51:51.087ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_5dda65cf-da71-4547-b51e-e1cf5d91d287") failed: Property not found. retry in 1.481043355s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_5dda65cf-da71-4547-b51e-e1cf5d91d287
10002023-11-20T07:51:51.106ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_8cb95d51-90ee-4878-942d-5d274a6755b3") failed: Property not found. retry in 1.015718707s file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_8cb95d51-90ee-4878-942d-5d274a6755b3
10012023-11-20T07:51:51.174ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_088f787e-f34a-4c96-b5de-12e3e6780164") failed: Property not found. retry in 1.155275418s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_088f787e-f34a-4c96-b5de-12e3e6780164
10022023-11-20T07:51:51.202ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_381f1240-83d8-4fb0-9bf7-1e162996eca3") failed: Property not found. retry in 212.391049ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_381f1240-83d8-4fb0-9bf7-1e162996eca3
10032023-11-20T07:51:51.307ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_84037cc2-9a57-4b92-aa79-1df7e38b4b09") failed: Property not found. retry in 1.366606982s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_84037cc2-9a57-4b92-aa79-1df7e38b4b09
10042023-11-20T07:51:51.331ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_05afc584-303a-494e-958d-48a2486cccde") failed: Property not found. retry in 924.86692ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_05afc584-303a-494e-958d-48a2486cccde
10052023-11-20T07:51:51.352ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_f8c559b1-5e66-4e93-9478-2feb1a69060f") failed: Property not found. retry in 507.580547ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_f8c559b1-5e66-4e93-9478-2feb1a69060f
10062023-11-20T07:51:51.356ZINFOSledAgent (ServiceManager): Profile for oxz_clickhouse_847cd99c-7101-48eb-bf5f-29a882b481fe: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/clickhouse"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value="oxControlService14"/> <propval type="astring" name="gateway" value="fd00:1122:3344:101::1"/> <propval type="astring" name="listen_addr" value="fd00:1122:3344:101::e"/> <propval type="astring" name="listen_port" value="8123"/> <propval type="astring" name="store" value="/data"/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/install"> <property_group type="application" name="install_props"> <property type="net_address" name="nameserver"> <net_address_list> <value_node value="fd00:1122:3344:3::1" /> <value_node value="fd00:1122:3344:1::1" /> <value_node value="fd00:1122:3344:2::1" /> </net_address_list> </property> </property_group> <instance enabled="true" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
10072023-11-20T07:51:51.356ZINFOSledAgent (ServiceManager): Zone booting file = illumos-utils/src/running_zone.rs:509 zone = oxz_clickhouse_847cd99c-7101-48eb-bf5f-29a882b481fe
10082023-11-20T07:51:51.398ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_8bbe15f8-a976-42cd-ae90-1c7c4c5d3fea") failed: Property not found. retry in 1.069888628s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_8bbe15f8-a976-42cd-ae90-1c7c4c5d3fea
10092023-11-20T07:51:51.435ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_381f1240-83d8-4fb0-9bf7-1e162996eca3") failed: Property not found. retry in 446.599788ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_381f1240-83d8-4fb0-9bf7-1e162996eca3
10102023-11-20T07:51:51.613ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_a9743596-df71-421e-b6dc-00d0636ce520") failed: Property not found. retry in 658.97584ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_a9743596-df71-421e-b6dc-00d0636ce520
10112023-11-20T07:51:51.636ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_07e03e9a-beed-4e41-b732-48a7946b5a69") failed: Property not found. retry in 1.105739919s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_07e03e9a-beed-4e41-b732-48a7946b5a69
10122023-11-20T07:51:51.656ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_249188b9-13b4-4193-9568-84db6fc17529") failed: Property not found. retry in 587.992955ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_249188b9-13b4-4193-9568-84db6fc17529
10132023-11-20T07:51:51.692ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_f17d1012-45a4-4e32-a4cd-f647739a92e6") failed: Property not found. retry in 1.296064143s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_f17d1012-45a4-4e32-a4cd-f647739a92e6
10142023-11-20T07:51:51.773ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_847cd99c-7101-48eb-bf5f-29a882b481fe") failed: Property not found. retry in 60.674329ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_847cd99c-7101-48eb-bf5f-29a882b481fe
10152023-11-20T07:51:51.825ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_dc808808-c29f-4e50-89dc-eb71dee374d3") failed: Property not found. retry in 542.262969ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_dc808808-c29f-4e50-89dc-eb71dee374d3
10162023-11-20T07:51:51.852ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_847cd99c-7101-48eb-bf5f-29a882b481fe") failed: Property not found. retry in 114.357187ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_847cd99c-7101-48eb-bf5f-29a882b481fe
10172023-11-20T07:51:51.922ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_16b5abb9-d189-4e4e-96c0-108193957b3d") failed: Property not found. retry in 1.331824293s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_16b5abb9-d189-4e4e-96c0-108193957b3d
10182023-11-20T07:51:51.943ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_f8c559b1-5e66-4e93-9478-2feb1a69060f") failed: Property not found. retry in 1.348485217s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_f8c559b1-5e66-4e93-9478-2feb1a69060f
10192023-11-20T07:51:51.959ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_381f1240-83d8-4fb0-9bf7-1e162996eca3") failed: Property not found. retry in 1.161291545s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_381f1240-83d8-4fb0-9bf7-1e162996eca3
10202023-11-20T07:51:51.974ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_201086a7-56f2-4de0-8aee-210e5cfefed7") failed: Property not found. retry in 818.276468ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_201086a7-56f2-4de0-8aee-210e5cfefed7
10212023-11-20T07:51:51.987ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_847cd99c-7101-48eb-bf5f-29a882b481fe") failed: Property not found. retry in 240.453312ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_847cd99c-7101-48eb-bf5f-29a882b481fe
10222023-11-20T07:51:52.008ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_3abd98fd-69cc-4f10-9866-db07b595b88e") failed: Property not found. retry in 1.2399806s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_3abd98fd-69cc-4f10-9866-db07b595b88e
10232023-11-20T07:51:52.141ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_8cb95d51-90ee-4878-942d-5d274a6755b3") failed: Property not found. retry in 1.223687936s file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_8cb95d51-90ee-4878-942d-5d274a6755b3
10242023-11-20T07:51:52.249ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_847cd99c-7101-48eb-bf5f-29a882b481fe") failed: Property not found. retry in 443.935985ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_847cd99c-7101-48eb-bf5f-29a882b481fe
10252023-11-20T07:51:52.267ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_249188b9-13b4-4193-9568-84db6fc17529") failed: Property not found. retry in 1.219343708s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_249188b9-13b4-4193-9568-84db6fc17529
10262023-11-20T07:51:52.285ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_05afc584-303a-494e-958d-48a2486cccde") failed: Property not found. retry in 1.007755558s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_05afc584-303a-494e-958d-48a2486cccde
10272023-11-20T07:51:52.303ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_a9743596-df71-421e-b6dc-00d0636ce520") failed: Property not found. retry in 758.746557ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_a9743596-df71-421e-b6dc-00d0636ce520
10282023-11-20T07:51:52.333ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_7ac38bb8-58b9-46d6-877a-58d68deadc2c") failed: Property not found. retry in 590.73646ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_7ac38bb8-58b9-46d6-877a-58d68deadc2c
10292023-11-20T07:51:52.351ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_088f787e-f34a-4c96-b5de-12e3e6780164") failed: Property not found. retry in 762.740083ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_088f787e-f34a-4c96-b5de-12e3e6780164
10302023-11-20T07:51:52.385ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_dc808808-c29f-4e50-89dc-eb71dee374d3") failed: Property not found. retry in 1.355610847s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_dc808808-c29f-4e50-89dc-eb71dee374d3
10312023-11-20T07:51:52.415ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_8a5fa6e3-3703-4684-a6e8-b41d55048601") failed: Property not found. retry in 1.216439474s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_8a5fa6e3-3703-4684-a6e8-b41d55048601
10322023-11-20T07:51:52.485ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_8bbe15f8-a976-42cd-ae90-1c7c4c5d3fea") failed: Property not found. retry in 1.316709503s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_8bbe15f8-a976-42cd-ae90-1c7c4c5d3fea
10332023-11-20T07:51:52.585ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_5dda65cf-da71-4547-b51e-e1cf5d91d287") failed: Property not found. retry in 596.734668ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_5dda65cf-da71-4547-b51e-e1cf5d91d287
10342023-11-20T07:51:52.690ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_84037cc2-9a57-4b92-aa79-1df7e38b4b09") failed: Property not found. retry in 788.411505ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_84037cc2-9a57-4b92-aa79-1df7e38b4b09
10352023-11-20T07:51:52.706ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_847cd99c-7101-48eb-bf5f-29a882b481fe") failed: Property not found. retry in 677.694359ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_847cd99c-7101-48eb-bf5f-29a882b481fe
10362023-11-20T07:51:52.757ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_07e03e9a-beed-4e41-b732-48a7946b5a69") failed: Property not found. retry in 1.406486791s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_07e03e9a-beed-4e41-b732-48a7946b5a69
10372023-11-20T07:51:52.809ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_201086a7-56f2-4de0-8aee-210e5cfefed7") failed: Property not found. retry in 1.14080597s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_201086a7-56f2-4de0-8aee-210e5cfefed7
10382023-11-20T07:51:52.897ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::f858:c2ff:fe1d:5103": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
10392023-11-20T07:51:52.940ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_7ac38bb8-58b9-46d6-877a-58d68deadc2c") failed: Property not found. retry in 1.489908901s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_7ac38bb8-58b9-46d6-877a-58d68deadc2c
10402023-11-20T07:51:53.004ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_f17d1012-45a4-4e32-a4cd-f647739a92e6") failed: Property not found. retry in 1.42837601s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_f17d1012-45a4-4e32-a4cd-f647739a92e6
10412023-11-20T07:51:53.075ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_a9743596-df71-421e-b6dc-00d0636ce520") failed: Property not found. retry in 1.20411733s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_a9743596-df71-421e-b6dc-00d0636ce520
10422023-11-20T07:51:53.129ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_088f787e-f34a-4c96-b5de-12e3e6780164") failed: Property not found. retry in 798.138492ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_088f787e-f34a-4c96-b5de-12e3e6780164
10432023-11-20T07:51:53.141ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_381f1240-83d8-4fb0-9bf7-1e162996eca3") failed: Property not found. retry in 590.899495ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_381f1240-83d8-4fb0-9bf7-1e162996eca3
10442023-11-20T07:51:53.196ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_5dda65cf-da71-4547-b51e-e1cf5d91d287") failed: Property not found. retry in 716.429368ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_5dda65cf-da71-4547-b51e-e1cf5d91d287
10452023-11-20T07:51:53.260ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_3abd98fd-69cc-4f10-9866-db07b595b88e") failed: Property not found. retry in 827.808207ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_3abd98fd-69cc-4f10-9866-db07b595b88e
10462023-11-20T07:51:53.272ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_16b5abb9-d189-4e4e-96c0-108193957b3d") failed: Property not found. retry in 501.285173ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_16b5abb9-d189-4e4e-96c0-108193957b3d
10472023-11-20T07:51:53.304ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_f8c559b1-5e66-4e93-9478-2feb1a69060f") failed: Property not found. retry in 739.022254ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_f8c559b1-5e66-4e93-9478-2feb1a69060f
10482023-11-20T07:51:53.317ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_05afc584-303a-494e-958d-48a2486cccde") failed: Property not found. retry in 688.277908ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_05afc584-303a-494e-958d-48a2486cccde
10492023-11-20T07:51:53.378ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_8cb95d51-90ee-4878-942d-5d274a6755b3") failed: Property not found. retry in 672.380729ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_8cb95d51-90ee-4878-942d-5d274a6755b3
10502023-11-20T07:51:53.409ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_847cd99c-7101-48eb-bf5f-29a882b481fe") failed: Property not found. retry in 1.153695238s file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_847cd99c-7101-48eb-bf5f-29a882b481fe
10512023-11-20T07:51:53.491ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_84037cc2-9a57-4b92-aa79-1df7e38b4b09") failed: Property not found. retry in 914.954747ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_84037cc2-9a57-4b92-aa79-1df7e38b4b09
10522023-11-20T07:51:53.503ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_249188b9-13b4-4193-9568-84db6fc17529") failed: Property not found. retry in 1.345824183s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_249188b9-13b4-4193-9568-84db6fc17529
10532023-11-20T07:51:53.646ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_8a5fa6e3-3703-4684-a6e8-b41d55048601") failed: Property not found. retry in 1.271206551s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_8a5fa6e3-3703-4684-a6e8-b41d55048601
10542023-11-20T07:51:53.744ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_381f1240-83d8-4fb0-9bf7-1e162996eca3") failed: Property not found. retry in 1.277002637s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_381f1240-83d8-4fb0-9bf7-1e162996eca3
10552023-11-20T07:51:53.755ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_dc808808-c29f-4e50-89dc-eb71dee374d3") failed: Property not found. retry in 1.460583436s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_dc808808-c29f-4e50-89dc-eb71dee374d3
10562023-11-20T07:51:53.786ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_16b5abb9-d189-4e4e-96c0-108193957b3d") failed: Property not found. retry in 1.219894542s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_16b5abb9-d189-4e4e-96c0-108193957b3d
10572023-11-20T07:51:53.816ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_8bbe15f8-a976-42cd-ae90-1c7c4c5d3fea") failed: Property not found. retry in 653.202469ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_8bbe15f8-a976-42cd-ae90-1c7c4c5d3fea
10582023-11-20T07:51:53.926ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_5dda65cf-da71-4547-b51e-e1cf5d91d287") failed: Property not found. retry in 783.497352ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_5dda65cf-da71-4547-b51e-e1cf5d91d287
10592023-11-20T07:51:53.940ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_088f787e-f34a-4c96-b5de-12e3e6780164") failed: Property not found. retry in 904.188819ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_088f787e-f34a-4c96-b5de-12e3e6780164
10602023-11-20T07:51:53.963ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_201086a7-56f2-4de0-8aee-210e5cfefed7") failed: Property not found. retry in 508.316132ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_201086a7-56f2-4de0-8aee-210e5cfefed7
10612023-11-20T07:51:54.026ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_05afc584-303a-494e-958d-48a2486cccde") failed: Property not found. retry in 984.604441ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_05afc584-303a-494e-958d-48a2486cccde
10622023-11-20T07:51:54.059ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_f8c559b1-5e66-4e93-9478-2feb1a69060f") failed: Property not found. retry in 759.586824ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_f8c559b1-5e66-4e93-9478-2feb1a69060f
10632023-11-20T07:51:54.070ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_8cb95d51-90ee-4878-942d-5d274a6755b3") failed: Property not found. retry in 1.040726782s file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_8cb95d51-90ee-4878-942d-5d274a6755b3
10642023-11-20T07:51:54.104ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_3abd98fd-69cc-4f10-9866-db07b595b88e") failed: Property not found. retry in 1.418554369s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_3abd98fd-69cc-4f10-9866-db07b595b88e
10652023-11-20T07:51:54.179ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_07e03e9a-beed-4e41-b732-48a7946b5a69") failed: Property not found. retry in 882.548443ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_07e03e9a-beed-4e41-b732-48a7946b5a69
10662023-11-20T07:51:54.296ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_a9743596-df71-421e-b6dc-00d0636ce520") failed: Property not found. retry in 1.003539032s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_a9743596-df71-421e-b6dc-00d0636ce520
10672023-11-20T07:51:54.420ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_84037cc2-9a57-4b92-aa79-1df7e38b4b09") failed: Property not found. retry in 1.287458637s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_84037cc2-9a57-4b92-aa79-1df7e38b4b09
10682023-11-20T07:51:54.444ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_7ac38bb8-58b9-46d6-877a-58d68deadc2c") failed: Property not found. retry in 1.018591462s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_7ac38bb8-58b9-46d6-877a-58d68deadc2c
10692023-11-20T07:51:54.457ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_f17d1012-45a4-4e32-a4cd-f647739a92e6") failed: Property not found. retry in 641.300369ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_f17d1012-45a4-4e32-a4cd-f647739a92e6
10702023-11-20T07:51:54.483ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_8bbe15f8-a976-42cd-ae90-1c7c4c5d3fea") failed: Property not found. retry in 1.139489693s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_8bbe15f8-a976-42cd-ae90-1c7c4c5d3fea
10712023-11-20T07:51:54.496ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_201086a7-56f2-4de0-8aee-210e5cfefed7") failed: Property not found. retry in 731.058333ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_201086a7-56f2-4de0-8aee-210e5cfefed7
10722023-11-20T07:51:54.579ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_847cd99c-7101-48eb-bf5f-29a882b481fe") failed: Property not found. retry in 948.74036ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_847cd99c-7101-48eb-bf5f-29a882b481fe
10732023-11-20T07:51:54.724ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_5dda65cf-da71-4547-b51e-e1cf5d91d287") failed: Property not found. retry in 511.983372ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_5dda65cf-da71-4547-b51e-e1cf5d91d287
10742023-11-20T07:51:54.831ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_f8c559b1-5e66-4e93-9478-2feb1a69060f") failed: Property not found. retry in 856.740119ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_f8c559b1-5e66-4e93-9478-2feb1a69060f
10752023-11-20T07:51:54.858ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_088f787e-f34a-4c96-b5de-12e3e6780164") failed: Property not found. retry in 1.384635109s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_088f787e-f34a-4c96-b5de-12e3e6780164
10762023-11-20T07:51:54.868ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_249188b9-13b4-4193-9568-84db6fc17529") failed: Property not found. retry in 789.07343ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_249188b9-13b4-4193-9568-84db6fc17529
10772023-11-20T07:51:54.931ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_8a5fa6e3-3703-4684-a6e8-b41d55048601") failed: Property not found. retry in 773.191087ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_8a5fa6e3-3703-4684-a6e8-b41d55048601
10782023-11-20T07:51:55.022ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_16b5abb9-d189-4e4e-96c0-108193957b3d") failed: Property not found. retry in 635.397743ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_16b5abb9-d189-4e4e-96c0-108193957b3d
10792023-11-20T07:51:55.036ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_05afc584-303a-494e-958d-48a2486cccde") failed: Property not found. retry in 1.252431437s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_05afc584-303a-494e-958d-48a2486cccde
10802023-11-20T07:51:55.048ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_381f1240-83d8-4fb0-9bf7-1e162996eca3") failed: Property not found. retry in 1.173763526s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_381f1240-83d8-4fb0-9bf7-1e162996eca3
10812023-11-20T07:51:55.075ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_07e03e9a-beed-4e41-b732-48a7946b5a69") failed: Property not found. retry in 502.778346ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_07e03e9a-beed-4e41-b732-48a7946b5a69
10822023-11-20T07:51:55.115ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_f17d1012-45a4-4e32-a4cd-f647739a92e6") failed: Property not found. retry in 1.493832561s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_f17d1012-45a4-4e32-a4cd-f647739a92e6
10832023-11-20T07:51:55.127ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_8cb95d51-90ee-4878-942d-5d274a6755b3") failed: Property not found. retry in 677.596579ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_8cb95d51-90ee-4878-942d-5d274a6755b3
10842023-11-20T07:51:55.236ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_dc808808-c29f-4e50-89dc-eb71dee374d3") failed: Property not found. retry in 891.4187ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_dc808808-c29f-4e50-89dc-eb71dee374d3
10852023-11-20T07:51:55.253ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_201086a7-56f2-4de0-8aee-210e5cfefed7") failed: Property not found. retry in 1.133357325s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_201086a7-56f2-4de0-8aee-210e5cfefed7
10862023-11-20T07:51:55.268ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_5dda65cf-da71-4547-b51e-e1cf5d91d287") failed: Property not found. retry in 1.196386388s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_5dda65cf-da71-4547-b51e-e1cf5d91d287
10872023-11-20T07:51:55.318ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_a9743596-df71-421e-b6dc-00d0636ce520") failed: Property not found. retry in 955.896239ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_a9743596-df71-421e-b6dc-00d0636ce520
10882023-11-20T07:51:55.477ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_7ac38bb8-58b9-46d6-877a-58d68deadc2c") failed: Property not found. retry in 861.063342ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_7ac38bb8-58b9-46d6-877a-58d68deadc2c
10892023-11-20T07:51:55.541ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_3abd98fd-69cc-4f10-9866-db07b595b88e") failed: Property not found. retry in 813.440977ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_3abd98fd-69cc-4f10-9866-db07b595b88e
10902023-11-20T07:51:55.554ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_847cd99c-7101-48eb-bf5f-29a882b481fe") failed: Property not found. retry in 1.156750036s file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_847cd99c-7101-48eb-bf5f-29a882b481fe
10912023-11-20T07:51:55.594ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_07e03e9a-beed-4e41-b732-48a7946b5a69") failed: Property not found. retry in 1.367747553s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_07e03e9a-beed-4e41-b732-48a7946b5a69
10922023-11-20T07:51:55.640ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_8bbe15f8-a976-42cd-ae90-1c7c4c5d3fea") failed: Property not found. retry in 792.465914ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_8bbe15f8-a976-42cd-ae90-1c7c4c5d3fea
10932023-11-20T07:51:55.708ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_249188b9-13b4-4193-9568-84db6fc17529") failed: Property not found. retry in 753.507729ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_249188b9-13b4-4193-9568-84db6fc17529
10942023-11-20T07:51:55.731ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_84037cc2-9a57-4b92-aa79-1df7e38b4b09") failed: Property not found. retry in 1.201481199s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_84037cc2-9a57-4b92-aa79-1df7e38b4b09
10952023-11-20T07:51:55.750ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_f8c559b1-5e66-4e93-9478-2feb1a69060f") failed: Property not found. retry in 1.093415389s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_f8c559b1-5e66-4e93-9478-2feb1a69060f
10962023-11-20T07:51:55.772ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_8a5fa6e3-3703-4684-a6e8-b41d55048601") failed: Property not found. retry in 614.818886ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_8a5fa6e3-3703-4684-a6e8-b41d55048601
10972023-11-20T07:51:55.902ZINFOSledAgent (ServiceManager): Ensuring address fd00:1122:3344:101::d exists file = sled-agent/src/services.rs:1370
10982023-11-20T07:51:55.902ZINFOSledAgent (ServiceManager): Adding address: Static(V6(Ipv6Network { addr: fd00:1122:3344:101::d, prefix: 64 })) file = illumos-utils/src/running_zone.rs:605 zone = oxz_oximeter_8cb95d51-90ee-4878-942d-5d274a6755b3
10992023-11-20T07:51:56.557ZINFOSledAgent (ServiceManager): Ensuring address fd00:1122:3344:101::d exists - OK file = sled-agent/src/services.rs:1378
11002023-11-20T07:51:56.557ZINFOSledAgent (ServiceManager): Zone using sled underlay as gateway file = sled-agent/src/services.rs:1399
11012023-11-20T07:51:57.681ZINFOSledAgent (ServiceManager): Setting up oximeter service file = sled-agent/src/services.rs:1639
11022023-11-20T07:51:57.818ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_a9743596-df71-421e-b6dc-00d0636ce520") failed: Property not found. retry in 813.457243ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_a9743596-df71-421e-b6dc-00d0636ce520
11032023-11-20T07:51:57.884ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_381f1240-83d8-4fb0-9bf7-1e162996eca3") failed: Property not found. retry in 760.482561ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_381f1240-83d8-4fb0-9bf7-1e162996eca3
11042023-11-20T07:51:57.899ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::f858:c2ff:fe1d:5103": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
11052023-11-20T07:51:57.949ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_847cd99c-7101-48eb-bf5f-29a882b481fe") failed: Property not found. retry in 1.377543363s file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_847cd99c-7101-48eb-bf5f-29a882b481fe
11062023-11-20T07:51:57.963ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_f8c559b1-5e66-4e93-9478-2feb1a69060f") failed: Property not found. retry in 1.311355002s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_f8c559b1-5e66-4e93-9478-2feb1a69060f
11072023-11-20T07:51:58.148ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_201086a7-56f2-4de0-8aee-210e5cfefed7") failed: Property not found. retry in 1.41624969s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_201086a7-56f2-4de0-8aee-210e5cfefed7
11082023-11-20T07:51:58.252ZINFOSledAgent (ServiceManager): Ensuring address fd00:1122:3344:101::8 exists file = sled-agent/src/services.rs:1370
11092023-11-20T07:51:58.252ZINFOSledAgent (ServiceManager): Adding address: Static(V6(Ipv6Network { addr: fd00:1122:3344:101::8, prefix: 64 })) file = illumos-utils/src/running_zone.rs:605 zone = oxz_external_dns_84037cc2-9a57-4b92-aa79-1df7e38b4b09
11102023-11-20T07:51:58.893ZINFOSledAgent (ServiceManager): Ensuring address fd00:1122:3344:101::8 exists - OK file = sled-agent/src/services.rs:1378
11112023-11-20T07:51:58.893ZINFOSledAgent (ServiceManager): Zone using sled underlay as gateway file = sled-agent/src/services.rs:1399
11122023-11-20T07:51:59.599ZINFOSledAgent (ServiceManager): Setting up external-dns service file = sled-agent/src/services.rs:1540
11132023-11-20T07:51:59.599ZINFOSledAgent (ServiceManager): Ensuring address for OPTE port file = illumos-utils/src/running_zone.rs:649 zone = oxz_external_dns_84037cc2-9a57-4b92-aa79-1df7e38b4b09
11142023-11-20T07:52:02.901ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::f858:c2ff:fe1d:5103": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
11152023-11-20T07:52:05.336ZINFOSledAgent (ServiceManager): Ensuring address fd00:1122:3344:101::9 exists file = sled-agent/src/services.rs:1370
11162023-11-20T07:52:05.336ZINFOSledAgent (ServiceManager): Adding address: Static(V6(Ipv6Network { addr: fd00:1122:3344:101::9, prefix: 64 })) file = illumos-utils/src/running_zone.rs:605 zone = oxz_external_dns_5dda65cf-da71-4547-b51e-e1cf5d91d287
11172023-11-20T07:52:05.891ZINFOSledAgent (ServiceManager): Ensuring address fd00:1122:3344:101::9 exists - OK file = sled-agent/src/services.rs:1378
11182023-11-20T07:52:05.891ZINFOSledAgent (ServiceManager): Zone using sled underlay as gateway file = sled-agent/src/services.rs:1399
11192023-11-20T07:52:06.083ZINFOSledAgent (ServiceManager): Setting up external-dns service file = sled-agent/src/services.rs:1540
11202023-11-20T07:52:06.084ZINFOSledAgent (ServiceManager): Ensuring address for OPTE port file = illumos-utils/src/running_zone.rs:649 zone = oxz_external_dns_5dda65cf-da71-4547-b51e-e1cf5d91d287
11212023-11-20T07:52:07.897ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::f858:c2ff:fe1d:5103": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
11222023-11-20T07:52:11.362ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:769 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:50229
11232023-11-20T07:52:11.362ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 63 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:50229 req_id = 364d045d-3cc7-40a1-8b8b-c285caa32e1c response_code = 200 uri = /baseboard
11242023-11-20T07:52:12.052ZINFOSledAgent (ServiceManager): Ensuring address fd00:1122:3344:101::b exists file = sled-agent/src/services.rs:1370
11252023-11-20T07:52:12.052ZINFOSledAgent (ServiceManager): Adding address: Static(V6(Ipv6Network { addr: fd00:1122:3344:101::b, prefix: 64 })) file = illumos-utils/src/running_zone.rs:605 zone = oxz_nexus_dc808808-c29f-4e50-89dc-eb71dee374d3
11262023-11-20T07:52:12.607ZINFOSledAgent (ServiceManager): Ensuring address fd00:1122:3344:101::b exists - OK file = sled-agent/src/services.rs:1378
11272023-11-20T07:52:12.607ZINFOSledAgent (ServiceManager): Zone using sled underlay as gateway file = sled-agent/src/services.rs:1399
11282023-11-20T07:52:12.809ZINFOSledAgent (ServiceManager): Setting up Nexus service file = sled-agent/src/services.rs:1441
11292023-11-20T07:52:12.809ZINFOSledAgent (ServiceManager): Ensuring address for OPTE port file = illumos-utils/src/running_zone.rs:649 zone = oxz_nexus_dc808808-c29f-4e50-89dc-eb71dee374d3
11302023-11-20T07:52:12.897ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::f858:c2ff:fe1d:5103": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
11312023-11-20T07:52:17.899ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::f858:c2ff:fe1d:5103": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
11322023-11-20T07:52:19.222ZINFOSledAgent (ServiceManager): Ensuring address fd00:1122:3344:101::a exists file = sled-agent/src/services.rs:1370
11332023-11-20T07:52:19.222ZINFOSledAgent (ServiceManager): Adding address: Static(V6(Ipv6Network { addr: fd00:1122:3344:101::a, prefix: 64 })) file = illumos-utils/src/running_zone.rs:605 zone = oxz_nexus_8a5fa6e3-3703-4684-a6e8-b41d55048601
11342023-11-20T07:52:19.781ZINFOSledAgent (ServiceManager): Ensuring address fd00:1122:3344:101::a exists - OK file = sled-agent/src/services.rs:1378
11352023-11-20T07:52:19.781ZINFOSledAgent (ServiceManager): Zone using sled underlay as gateway file = sled-agent/src/services.rs:1399
11362023-11-20T07:52:19.986ZINFOSledAgent (ServiceManager): Setting up Nexus service file = sled-agent/src/services.rs:1441
11372023-11-20T07:52:19.987ZINFOSledAgent (ServiceManager): Ensuring address for OPTE port file = illumos-utils/src/running_zone.rs:649 zone = oxz_nexus_8a5fa6e3-3703-4684-a6e8-b41d55048601
11382023-11-20T07:52:22.900ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::f858:c2ff:fe1d:5103": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
11392023-11-20T07:52:25.459ZINFOSledAgent (ServiceManager): Ensuring address fd00:1122:3344:101::c exists file = sled-agent/src/services.rs:1370
11402023-11-20T07:52:25.459ZINFOSledAgent (ServiceManager): Adding address: Static(V6(Ipv6Network { addr: fd00:1122:3344:101::c, prefix: 64 })) file = illumos-utils/src/running_zone.rs:605 zone = oxz_nexus_381f1240-83d8-4fb0-9bf7-1e162996eca3
11412023-11-20T07:52:26.015ZINFOSledAgent (ServiceManager): Ensuring address fd00:1122:3344:101::c exists - OK file = sled-agent/src/services.rs:1378
11422023-11-20T07:52:26.015ZINFOSledAgent (ServiceManager): Zone using sled underlay as gateway file = sled-agent/src/services.rs:1399
11432023-11-20T07:52:26.233ZINFOSledAgent (ServiceManager): Setting up Nexus service file = sled-agent/src/services.rs:1441
11442023-11-20T07:52:26.233ZINFOSledAgent (ServiceManager): Ensuring address for OPTE port file = illumos-utils/src/running_zone.rs:649 zone = oxz_nexus_381f1240-83d8-4fb0-9bf7-1e162996eca3
11452023-11-20T07:52:27.902ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::f858:c2ff:fe1d:5103": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
11462023-11-20T07:52:31.224ZINFOSledAgent (ServiceManager): Writing ledger to /pool/int/a462a7f7-b628-40fe-80ff-4e4189e2d62b/config/.services.json.tmp file = common/src/ledger.rs:197
11472023-11-20T07:52:31.225ZINFOSledAgent (ServiceManager): Writing ledger to /pool/int/b462a7f7-b628-40fe-80ff-4e4189e2d62b/config/.services.json.tmp file = common/src/ledger.rs:197
11482023-11-20T07:52:31.226ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 49865374 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::1]:65090 req_id = 6c031154-e3a8-4e11-9a10-ff0e8b5b1e9d response_code = 204 uri = /services
11492023-11-20T07:52:31.229ZINFOSledAgent (RSS): Finished setting up services file = sled-agent/src/rack_setup/service.rs:1008
11502023-11-20T07:52:31.229ZINFOSledAgent (RSS): Writing ledger to /pool/int/a462a7f7-b628-40fe-80ff-4e4189e2d62b/config/.rss-plan-completed.marker.tmp file = common/src/ledger.rs:197
11512023-11-20T07:52:31.229ZINFOSledAgent (RSS): Writing ledger to /pool/int/b462a7f7-b628-40fe-80ff-4e4189e2d62b/config/.rss-plan-completed.marker.tmp file = common/src/ledger.rs:197
11522023-11-20T07:52:31.232ZINFOSledAgent (RSS): Handing off control to Nexus file = sled-agent/src/rack_setup/service.rs:525
11532023-11-20T07:52:31.232ZINFOSledAgent (RSS): Nexus address: [fd00:1122:3344:101::c]:12221 file = sled-agent/src/rack_setup/service.rs:527
11542023-11-20T07:52:31.264ZINFOSledAgent (RSS): rack_network_config: Some( RackNetworkConfigV1 { bgp: [], infra_ip_first: 10.151.2.174, infra_ip_last: 10.151.2.174, ports: [ PortConfigV1 { addresses: [ V4( Ipv4Network { addr: 10.151.2.174, prefix: 32, }, ), ], bgp_peers: [], port: "qsfp0", routes: [ RouteConfig { destination: V4( Ipv4Network { addr: 0.0.0.0, prefix: 0, }, ), nexthop: 10.151.2.100, }, ], switch: Switch0, uplink_port_fec: None, uplink_port_speed: Speed40G, }, ], rack_subnet: Ipv6Network { addr: fd00:1122:3344:1::, prefix: 56, }, }, ) file = sled-agent/src/rack_setup/service.rs:639
11552023-11-20T07:52:31.265ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/racks/e54261f8-48ed-419b-92cd-78ba43007b3f/initialization-complete): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/rack_setup/service.rs:660
11562023-11-20T07:52:31.328ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/racks/e54261f8-48ed-419b-92cd-78ba43007b3f/initialization-complete): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/rack_setup/service.rs:660
11572023-11-20T07:52:31.433ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/racks/e54261f8-48ed-419b-92cd-78ba43007b3f/initialization-complete): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/rack_setup/service.rs:660
11582023-11-20T07:52:31.530ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/racks/e54261f8-48ed-419b-92cd-78ba43007b3f/initialization-complete): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/rack_setup/service.rs:660
11592023-11-20T07:52:32.770ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "82e80d1c-f9be-4d62-b6de-3996c8d37c41", "content-length": "124", "date": "Mon, 20 Nov 2023 07:52:32 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "82e80d1c-f9be-4d62-b6de-3996c8d37c41" } file = sled-agent/src/rack_setup/service.rs:660
11602023-11-20T07:52:32.904ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::f858:c2ff:fe1d:5103": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
11612023-11-20T07:52:32.959ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "9e6be662-ed75-4282-bb01-ccd6e6c20917", "content-length": "124", "date": "Mon, 20 Nov 2023 07:52:32 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "9e6be662-ed75-4282-bb01-ccd6e6c20917" } file = sled-agent/src/rack_setup/service.rs:660
11622023-11-20T07:52:33.276ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "2eeab996-b76f-41b9-b425-ef85dd8b6668", "content-length": "124", "date": "Mon, 20 Nov 2023 07:52:33 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "2eeab996-b76f-41b9-b425-ef85dd8b6668" } file = sled-agent/src/rack_setup/service.rs:660
11632023-11-20T07:52:33.557ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "98c14fb8-5bbb-4295-87f2-376662cd8e4a", "content-length": "124", "date": "Mon, 20 Nov 2023 07:52:32 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "98c14fb8-5bbb-4295-87f2-376662cd8e4a" } file = sled-agent/src/rack_setup/service.rs:660
11642023-11-20T07:52:33.960ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "9104cd51-f5b4-4b67-8a39-b79f8d05ba18", "content-length": "124", "date": "Mon, 20 Nov 2023 07:52:33 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "9104cd51-f5b4-4b67-8a39-b79f8d05ba18" } file = sled-agent/src/rack_setup/service.rs:660
11652023-11-20T07:52:34.271ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "4d6ff789-4092-4c00-a464-c361ad80bc22", "content-length": "124", "date": "Mon, 20 Nov 2023 07:52:33 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "4d6ff789-4092-4c00-a464-c361ad80bc22" } file = sled-agent/src/rack_setup/service.rs:660
11662023-11-20T07:52:34.872ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "68c64b18-1e55-45b6-a01c-41d99a50e14c", "content-length": "124", "date": "Mon, 20 Nov 2023 07:52:34 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "68c64b18-1e55-45b6-a01c-41d99a50e14c" } file = sled-agent/src/rack_setup/service.rs:660
11672023-11-20T07:52:35.805ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "121597fa-569f-4cfb-ab63-a0a8f23e4468", "content-length": "124", "date": "Mon, 20 Nov 2023 07:52:35 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "121597fa-569f-4cfb-ab63-a0a8f23e4468" } file = sled-agent/src/rack_setup/service.rs:660
11682023-11-20T07:52:36.790ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "bf4397b5-095d-441e-9bcd-8f5190c533e0", "content-length": "124", "date": "Mon, 20 Nov 2023 07:52:36 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "bf4397b5-095d-441e-9bcd-8f5190c533e0" } file = sled-agent/src/rack_setup/service.rs:660
11692023-11-20T07:52:37.858ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "9874e642-d10c-4945-82b3-a579d4825d5f", "content-length": "124", "date": "Mon, 20 Nov 2023 07:52:37 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "9874e642-d10c-4945-82b3-a579d4825d5f" } file = sled-agent/src/rack_setup/service.rs:660
11702023-11-20T07:52:37.906ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::f858:c2ff:fe1d:5103": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
11712023-11-20T07:52:38.688ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "9dc58490-dc48-4ece-bd0c-76f840207377", "content-length": "124", "date": "Mon, 20 Nov 2023 07:52:38 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "9dc58490-dc48-4ece-bd0c-76f840207377" } file = sled-agent/src/rack_setup/service.rs:660
11722023-11-20T07:52:39.720ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "57a32e56-077e-40bb-afb1-cb70e7b578fe", "content-length": "124", "date": "Mon, 20 Nov 2023 07:52:39 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "57a32e56-077e-40bb-afb1-cb70e7b578fe" } file = sled-agent/src/rack_setup/service.rs:660
11732023-11-20T07:52:41.395ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:769 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:62828
11742023-11-20T07:52:41.395ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 38 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:62828 req_id = a0c972b5-b8f4-42ac-af07-846f476c6b0c response_code = 200 uri = /baseboard
11752023-11-20T07:52:41.625ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "dea421e6-65da-4aa5-9a30-ed5bd5624bce", "content-length": "124", "date": "Mon, 20 Nov 2023 07:52:41 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "dea421e6-65da-4aa5-9a30-ed5bd5624bce" } file = sled-agent/src/rack_setup/service.rs:660
11762023-11-20T07:52:42.906ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::f858:c2ff:fe1d:5103": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
11772023-11-20T07:52:43.297ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "68465f96-1b29-4d49-8750-c6d60d03ada6", "content-length": "124", "date": "Mon, 20 Nov 2023 07:52:43 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "68465f96-1b29-4d49-8750-c6d60d03ada6" } file = sled-agent/src/rack_setup/service.rs:660
11782023-11-20T07:52:46.025ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "b65a6ae4-0ca7-4589-ba37-a40c12dcb4f3", "content-length": "124", "date": "Mon, 20 Nov 2023 07:52:46 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "b65a6ae4-0ca7-4589-ba37-a40c12dcb4f3" } file = sled-agent/src/rack_setup/service.rs:660
11792023-11-20T07:52:47.908ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::f858:c2ff:fe1d:5103": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
11802023-11-20T07:52:49.537ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "c4415e2c-9c09-490c-b061-d19a670aeaf9", "content-length": "124", "date": "Mon, 20 Nov 2023 07:52:49 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "c4415e2c-9c09-490c-b061-d19a670aeaf9" } file = sled-agent/src/rack_setup/service.rs:660
11812023-11-20T07:52:50.668ZWARNSledAgent (StorageMonitor): failed to notify nexus about a new pool oxp_616b26df-e62a-4c68-b506-f4a923d8aaf7 on disk DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-616b26df-e62a-4c68-b506-f4a923d8aaf7", model: "synthetic-model" } err = "Error Response: status: 404 Not Found; headers: {\\"content-type\\": \\"application/json\\", \\"x-request-id\\": \\"81f9c04f-208f-41ec-babe-0cd80b4fed0b\\", \\"content-length\\": \\"272\\", \\"date\\": \\"Mon, 20 Nov 2023 07:52:50 GMT\\"}; value: Error { error_code: Some(\\"ObjectNotFound\\"), message: \\"not found: physical-disk with id \\\\\\"vendor = \\\\\\"synthetic-vendor\\\\\\", serial = \\\\\\"synthetic-serial-616b26df-e62a-4c68-b506-f4a923d8aaf7\\\\\\", model = \\\\\\"synthetic-model\\\\\\"\\\\\\"\\", request_id: \\"81f9c04f-208f-41ec-babe-0cd80b4fed0b\\" }" file = sled-agent/src/storage_monitor.rs:273 total duration = 155.914142991s
11822023-11-20T07:52:50.773ZINFOSledAgent (HardwareMonitor): contacting server nexus, registering sled baseboard = Baseboard { part_number: "i86pc", revision: 0, serial_number: "57495fb0328c443a91abff25dfea8602" } file = sled-agent/src/sled_agent.rs:626 id = e75545d5-2741-43ba-8c56-e7f2943a5b82
11832023-11-20T07:52:50.815ZWARNSledAgent (StorageMonitor): failed to notify nexus about a new pool oxi_b462a7f7-b628-40fe-80ff-4e4189e2d62b on disk DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-b462a7f7-b628-40fe-80ff-4e4189e2d62b", model: "synthetic-model" } err = "Error Response: status: 404 Not Found; headers: {\\"content-type\\": \\"application/json\\", \\"x-request-id\\": \\"99551a67-de9f-423d-970c-1f3af791eec9\\", \\"content-length\\": \\"272\\", \\"date\\": \\"Mon, 20 Nov 2023 07:52:50 GMT\\"}; value: Error { error_code: Some(\\"ObjectNotFound\\"), message: \\"not found: physical-disk with id \\\\\\"vendor = \\\\\\"synthetic-vendor\\\\\\", serial = \\\\\\"synthetic-serial-b462a7f7-b628-40fe-80ff-4e4189e2d62b\\\\\\", model = \\\\\\"synthetic-model\\\\\\"\\\\\\"\\", request_id: \\"99551a67-de9f-423d-970c-1f3af791eec9\\" }" file = sled-agent/src/storage_monitor.rs:273 total duration = 156.060987419s
11842023-11-20T07:52:50.854ZWARNSledAgent (StorageMonitor): failed to notify nexus about a new pool oxi_a462a7f7-b628-40fe-80ff-4e4189e2d62b on disk DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-a462a7f7-b628-40fe-80ff-4e4189e2d62b", model: "synthetic-model" } err = "Error Response: status: 404 Not Found; headers: {\\"content-type\\": \\"application/json\\", \\"x-request-id\\": \\"07422748-d57d-43f2-aaa0-812d956ddb17\\", \\"content-length\\": \\"272\\", \\"date\\": \\"Mon, 20 Nov 2023 07:52:50 GMT\\"}; value: Error { error_code: Some(\\"ObjectNotFound\\"), message: \\"not found: physical-disk with id \\\\\\"vendor = \\\\\\"synthetic-vendor\\\\\\", serial = \\\\\\"synthetic-serial-a462a7f7-b628-40fe-80ff-4e4189e2d62b\\\\\\", model = \\\\\\"synthetic-model\\\\\\"\\\\\\"\\", request_id: \\"07422748-d57d-43f2-aaa0-812d956ddb17\\" }" file = sled-agent/src/storage_monitor.rs:273 total duration = 156.100109301s
11852023-11-20T07:52:50.856ZWARNSledAgent (StorageMonitor): failed to notify nexus about a new pool oxp_24b4dc87-ab46-49fb-a4b4-d361ae214c03 on disk DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-24b4dc87-ab46-49fb-a4b4-d361ae214c03", model: "synthetic-model" } err = "Error Response: status: 404 Not Found; headers: {\\"content-type\\": \\"application/json\\", \\"x-request-id\\": \\"2bdb468f-5ab4-4862-8458-f3d043dbbbe5\\", \\"content-length\\": \\"272\\", \\"date\\": \\"Mon, 20 Nov 2023 07:52:50 GMT\\"}; value: Error { error_code: Some(\\"ObjectNotFound\\"), message: \\"not found: physical-disk with id \\\\\\"vendor = \\\\\\"synthetic-vendor\\\\\\", serial = \\\\\\"synthetic-serial-24b4dc87-ab46-49fb-a4b4-d361ae214c03\\\\\\", model = \\\\\\"synthetic-model\\\\\\"\\\\\\"\\", request_id: \\"2bdb468f-5ab4-4862-8458-f3d043dbbbe5\\" }" file = sled-agent/src/storage_monitor.rs:273 total duration = 156.102247282s
11862023-11-20T07:52:50.897ZWARNSledAgent (StorageMonitor): failed to notify nexus about a new pool oxp_e4b4dc87-ab46-49fb-a4b4-d361ae214c03 on disk DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-e4b4dc87-ab46-49fb-a4b4-d361ae214c03", model: "synthetic-model" } err = "Error Response: status: 404 Not Found; headers: {\\"content-type\\": \\"application/json\\", \\"x-request-id\\": \\"7acfc1d3-0046-4a85-8e19-15748db02c37\\", \\"content-length\\": \\"272\\", \\"date\\": \\"Mon, 20 Nov 2023 07:52:50 GMT\\"}; value: Error { error_code: Some(\\"ObjectNotFound\\"), message: \\"not found: physical-disk with id \\\\\\"vendor = \\\\\\"synthetic-vendor\\\\\\", serial = \\\\\\"synthetic-serial-e4b4dc87-ab46-49fb-a4b4-d361ae214c03\\\\\\", model = \\\\\\"synthetic-model\\\\\\"\\\\\\"\\", request_id: \\"7acfc1d3-0046-4a85-8e19-15748db02c37\\" }" file = sled-agent/src/storage_monitor.rs:273 total duration = 156.142446244s
11872023-11-20T07:52:50.935ZINFOSledAgent (StorageMonitor): Nexus notification complete: Put(PhysicalDiskPutRequest { model: "synthetic-model", serial: "synthetic-serial-14b4dc87-ab46-49fb-a4b4-d361ae214c03", sled_id: e75545d5-2741-43ba-8c56-e7f2943a5b82, variant: U2, vendor: "synthetic-vendor" }) file = sled-agent/src/storage_monitor.rs:105
11882023-11-20T07:52:50.971ZWARNSledAgent (StorageMonitor): failed to notify nexus about a new pool oxp_ceb4461c-cf56-4719-ad3c-14430bfdfb60 on disk DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-ceb4461c-cf56-4719-ad3c-14430bfdfb60", model: "synthetic-model" } err = "Error Response: status: 404 Not Found; headers: {\\"content-type\\": \\"application/json\\", \\"x-request-id\\": \\"87ed7f89-7e00-492a-873c-3ed2518dddac\\", \\"content-length\\": \\"272\\", \\"date\\": \\"Mon, 20 Nov 2023 07:52:50 GMT\\"}; value: Error { error_code: Some(\\"ObjectNotFound\\"), message: \\"not found: physical-disk with id \\\\\\"vendor = \\\\\\"synthetic-vendor\\\\\\", serial = \\\\\\"synthetic-serial-ceb4461c-cf56-4719-ad3c-14430bfdfb60\\\\\\", model = \\\\\\"synthetic-model\\\\\\"\\\\\\"\\", request_id: \\"87ed7f89-7e00-492a-873c-3ed2518dddac\\" }" file = sled-agent/src/storage_monitor.rs:273 total duration = 156.216163119s
11892023-11-20T07:52:50.994ZINFOSledAgent (StorageMonitor): Nexus notification complete: Put(PhysicalDiskPutRequest { model: "synthetic-model", serial: "synthetic-serial-24b4dc87-ab46-49fb-a4b4-d361ae214c03", sled_id: e75545d5-2741-43ba-8c56-e7f2943a5b82, variant: U2, vendor: "synthetic-vendor" }) file = sled-agent/src/storage_monitor.rs:105
11902023-11-20T07:52:50.994ZINFOSledAgent (StorageMonitor): Nexus notification complete: Put(PhysicalDiskPutRequest { model: "synthetic-model", serial: "synthetic-serial-31bd71cd-4736-4a12-a387-9b74b050396f", sled_id: e75545d5-2741-43ba-8c56-e7f2943a5b82, variant: U2, vendor: "synthetic-vendor" }) file = sled-agent/src/storage_monitor.rs:105
11912023-11-20T07:52:51.025ZINFOSledAgent (StorageMonitor): Nexus notification complete: Put(PhysicalDiskPutRequest { model: "synthetic-model", serial: "synthetic-serial-616b26df-e62a-4c68-b506-f4a923d8aaf7", sled_id: e75545d5-2741-43ba-8c56-e7f2943a5b82, variant: U2, vendor: "synthetic-vendor" }) file = sled-agent/src/storage_monitor.rs:105
11922023-11-20T07:52:51.025ZINFOSledAgent (StorageMonitor): Nexus notification complete: Put(PhysicalDiskPutRequest { model: "synthetic-model", serial: "synthetic-serial-a462a7f7-b628-40fe-80ff-4e4189e2d62b", sled_id: e75545d5-2741-43ba-8c56-e7f2943a5b82, variant: M2, vendor: "synthetic-vendor" }) file = sled-agent/src/storage_monitor.rs:105
11932023-11-20T07:52:51.049ZWARNSledAgent (StorageMonitor): failed to notify nexus about a new pool oxp_cd70d7f6-2354-4bf2-8012-55bf9eaf7930 on disk DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-cd70d7f6-2354-4bf2-8012-55bf9eaf7930", model: "synthetic-model" } err = "Error Response: status: 404 Not Found; headers: {\\"content-type\\": \\"application/json\\", \\"x-request-id\\": \\"712d98e8-b611-4195-8379-da3444163ce6\\", \\"content-length\\": \\"272\\", \\"date\\": \\"Mon, 20 Nov 2023 07:52:50 GMT\\"}; value: Error { error_code: Some(\\"ObjectNotFound\\"), message: \\"not found: physical-disk with id \\\\\\"vendor = \\\\\\"synthetic-vendor\\\\\\", serial = \\\\\\"synthetic-serial-cd70d7f6-2354-4bf2-8012-55bf9eaf7930\\\\\\", model = \\\\\\"synthetic-model\\\\\\"\\\\\\"\\", request_id: \\"712d98e8-b611-4195-8379-da3444163ce6\\" }" file = sled-agent/src/storage_monitor.rs:273 total duration = 156.294107216s
11942023-11-20T07:52:51.057ZINFOSledAgent (StorageMonitor): Nexus notification complete: Put(PhysicalDiskPutRequest { model: "synthetic-model", serial: "synthetic-serial-b462a7f7-b628-40fe-80ff-4e4189e2d62b", sled_id: e75545d5-2741-43ba-8c56-e7f2943a5b82, variant: M2, vendor: "synthetic-vendor" }) file = sled-agent/src/storage_monitor.rs:105
11952023-11-20T07:52:51.074ZINFOSledAgent (StorageMonitor): Nexus notification complete: Put(PhysicalDiskPutRequest { model: "synthetic-model", serial: "synthetic-serial-cd70d7f6-2354-4bf2-8012-55bf9eaf7930", sled_id: e75545d5-2741-43ba-8c56-e7f2943a5b82, variant: U2, vendor: "synthetic-vendor" }) file = sled-agent/src/storage_monitor.rs:105
11962023-11-20T07:52:51.074ZINFOSledAgent (StorageMonitor): Nexus notification complete: Put(PhysicalDiskPutRequest { model: "synthetic-model", serial: "synthetic-serial-ceb4461c-cf56-4719-ad3c-14430bfdfb60", sled_id: e75545d5-2741-43ba-8c56-e7f2943a5b82, variant: U2, vendor: "synthetic-vendor" }) file = sled-agent/src/storage_monitor.rs:105
11972023-11-20T07:52:51.074ZINFOSledAgent (StorageMonitor): Nexus notification complete: Put(PhysicalDiskPutRequest { model: "synthetic-model", serial: "synthetic-serial-d462a7f7-b628-40fe-80ff-4e4189e2d62b", sled_id: e75545d5-2741-43ba-8c56-e7f2943a5b82, variant: U2, vendor: "synthetic-vendor" }) file = sled-agent/src/storage_monitor.rs:105
11982023-11-20T07:52:51.074ZINFOSledAgent (StorageMonitor): Nexus notification complete: Put(PhysicalDiskPutRequest { model: "synthetic-model", serial: "synthetic-serial-e4b4dc87-ab46-49fb-a4b4-d361ae214c03", sled_id: e75545d5-2741-43ba-8c56-e7f2943a5b82, variant: U2, vendor: "synthetic-vendor" }) file = sled-agent/src/storage_monitor.rs:105
11992023-11-20T07:52:51.074ZINFOSledAgent (StorageMonitor): Nexus notification complete: Put(PhysicalDiskPutRequest { model: "synthetic-model", serial: "synthetic-serial-f4b4dc87-ab46-49fb-a4b4-d361ae214c03", sled_id: e75545d5-2741-43ba-8c56-e7f2943a5b82, variant: U2, vendor: "synthetic-vendor" }) file = sled-agent/src/storage_monitor.rs:105
12002023-11-20T07:52:51.074ZINFOSledAgent (StorageMonitor): Nexus notification complete: ZpoolPutRequest { disk_model: "synthetic-model", disk_serial: "synthetic-serial-14b4dc87-ab46-49fb-a4b4-d361ae214c03", disk_vendor: "synthetic-vendor", size: ByteCount(20937965568) } file = sled-agent/src/storage_monitor.rs:105
12012023-11-20T07:52:52.908ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::f858:c2ff:fe1d:5103": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
12022023-11-20T07:52:53.826ZINFOSledAgent (StorageMonitor): Nexus notification complete: ZpoolPutRequest { disk_model: "synthetic-model", disk_serial: "synthetic-serial-24b4dc87-ab46-49fb-a4b4-d361ae214c03", disk_vendor: "synthetic-vendor", size: ByteCount(20937965568) } file = sled-agent/src/storage_monitor.rs:105
12032023-11-20T07:52:53.826ZINFOSledAgent (StorageMonitor): Nexus notification complete: ZpoolPutRequest { disk_model: "synthetic-model", disk_serial: "synthetic-serial-31bd71cd-4736-4a12-a387-9b74b050396f", disk_vendor: "synthetic-vendor", size: ByteCount(20937965568) } file = sled-agent/src/storage_monitor.rs:105
12042023-11-20T07:52:54.595ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 404 Not Found; headers: {"content-type": "application/json", "x-request-id": "cb0338b1-6914-44ad-8977-d99a2abd5cf6", "content-length": "174", "date": "Mon, 20 Nov 2023 07:52:54 GMT"}; value: Error { error_code: Some("ObjectNotFound"), message: "not found: zpool with id \"616b26df-e62a-4c68-b506-f4a923d8aaf7\"", request_id: "cb0338b1-6914-44ad-8977-d99a2abd5cf6" } file = sled-agent/src/rack_setup/service.rs:660
12052023-11-20T07:52:55.443ZINFOSledAgent (StorageMonitor): Nexus notification complete: ZpoolPutRequest { disk_model: "synthetic-model", disk_serial: "synthetic-serial-616b26df-e62a-4c68-b506-f4a923d8aaf7", disk_vendor: "synthetic-vendor", size: ByteCount(20937965568) } file = sled-agent/src/storage_monitor.rs:105
12062023-11-20T07:52:55.443ZINFOSledAgent (StorageMonitor): Nexus notification complete: ZpoolPutRequest { disk_model: "synthetic-model", disk_serial: "synthetic-serial-a462a7f7-b628-40fe-80ff-4e4189e2d62b", disk_vendor: "synthetic-vendor", size: ByteCount(20937965568) } file = sled-agent/src/storage_monitor.rs:105
12072023-11-20T07:52:55.443ZINFOSledAgent (StorageMonitor): Nexus notification complete: ZpoolPutRequest { disk_model: "synthetic-model", disk_serial: "synthetic-serial-b462a7f7-b628-40fe-80ff-4e4189e2d62b", disk_vendor: "synthetic-vendor", size: ByteCount(20937965568) } file = sled-agent/src/storage_monitor.rs:105
12082023-11-20T07:52:55.443ZINFOSledAgent (StorageMonitor): Nexus notification complete: ZpoolPutRequest { disk_model: "synthetic-model", disk_serial: "synthetic-serial-cd70d7f6-2354-4bf2-8012-55bf9eaf7930", disk_vendor: "synthetic-vendor", size: ByteCount(20937965568) } file = sled-agent/src/storage_monitor.rs:105
12092023-11-20T07:52:56.221ZINFOSledAgent (StorageMonitor): Nexus notification complete: ZpoolPutRequest { disk_model: "synthetic-model", disk_serial: "synthetic-serial-ceb4461c-cf56-4719-ad3c-14430bfdfb60", disk_vendor: "synthetic-vendor", size: ByteCount(20937965568) } file = sled-agent/src/storage_monitor.rs:105
12102023-11-20T07:52:56.221ZINFOSledAgent (StorageMonitor): Nexus notification complete: ZpoolPutRequest { disk_model: "synthetic-model", disk_serial: "synthetic-serial-d462a7f7-b628-40fe-80ff-4e4189e2d62b", disk_vendor: "synthetic-vendor", size: ByteCount(20937965568) } file = sled-agent/src/storage_monitor.rs:105
12112023-11-20T07:52:56.221ZINFOSledAgent (StorageMonitor): Nexus notification complete: ZpoolPutRequest { disk_model: "synthetic-model", disk_serial: "synthetic-serial-e4b4dc87-ab46-49fb-a4b4-d361ae214c03", disk_vendor: "synthetic-vendor", size: ByteCount(20937965568) } file = sled-agent/src/storage_monitor.rs:105
12122023-11-20T07:52:56.221ZINFOSledAgent (StorageMonitor): Nexus notification complete: ZpoolPutRequest { disk_model: "synthetic-model", disk_serial: "synthetic-serial-f4b4dc87-ab46-49fb-a4b4-d361ae214c03", disk_vendor: "synthetic-vendor", size: ByteCount(20937965568) } file = sled-agent/src/storage_monitor.rs:105
12132023-11-20T07:52:57.909ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::f858:c2ff:fe1d:5103": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
12142023-11-20T07:52:59.189ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:769 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:61199
12152023-11-20T07:52:59.189ZINFOSledAgent (PortManager): Ensuring VPC firewall rules file = illumos-utils/src/opte/port_manager.rs:399 rules = [VpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: c01ef95f-95de-4126-8d67-460994581d78, kind: Service { id: 84037cc2-9a57-4b92-aa79-1df7e38b4b09 }, name: Name("external-dns-84037cc2-9a57-4b92-aa79-1df7e38b4b09"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 130, 203])), subnet: V4(Ipv4Net(Ipv4Network { addr: 172.30.1.0, prefix: 24 })), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: cc29b875-5948-412e-af7a-ce1c0c8e86ae, kind: Service { id: 5dda65cf-da71-4547-b51e-e1cf5d91d287 }, name: Name("external-dns-5dda65cf-da71-4547-b51e-e1cf5d91d287"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 248, 109])), subnet: V4(Ipv4Net(Ipv4Network { addr: 172.30.1.0, prefix: 24 })), vni: Vni(100), primary: true, slot: 0 }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, VpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 96cd7458-ba2f-4142-af49-fe66c0c7b8d4, kind: Service { id: dc808808-c29f-4e50-89dc-eb71dee374d3 }, name: Name("nexus-dc808808-c29f-4e50-89dc-eb71dee374d3"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 170, 153])), subnet: V4(Ipv4Net(Ipv4Network { addr: 172.30.2.0, prefix: 24 })), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: bb3f68ea-6a70-4785-aaee-607c1a5127cd, kind: Service { id: 381f1240-83d8-4fb0-9bf7-1e162996eca3 }, name: Name("nexus-381f1240-83d8-4fb0-9bf7-1e162996eca3"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 224, 138])), subnet: V4(Ipv4Net(Ipv4Network { addr: 172.30.2.0, prefix: 24 })), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: ceca0063-034f-4eaf-b17a-618175b29716, kind: Service { id: 8a5fa6e3-3703-4684-a6e8-b41d55048601 }, name: Name("nexus-8a5fa6e3-3703-4684-a6e8-b41d55048601"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 138, 70])), subnet: V4(Ipv4Net(Ipv4Network { addr: 172.30.2.0, prefix: 24 })), vni: Vni(100), primary: true, slot: 0 }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }] vni = Vni(100)
12162023-11-20T07:52:59.189ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:418 port = "opte0" rules = []
12172023-11-20T07:52:59.189ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:418 port = "opte1" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
12182023-11-20T07:52:59.189ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:418 port = "opte3" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
12192023-11-20T07:52:59.189ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:418 port = "opte2" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
12202023-11-20T07:52:59.189ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:418 port = "opte4" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
12212023-11-20T07:52:59.189ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:418 port = "opte5" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
12222023-11-20T07:52:59.189ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 391 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:61199 req_id = 0bb347ce-5d89-441a-b831-ea90957a15bb response_code = 204 uri = /vpc/001de000-074c-4000-8000-000000000000/firewall/rules
12232023-11-20T07:52:59.503ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:769 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:50914
12242023-11-20T07:52:59.503ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 90 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:50914 req_id = b9550ac3-f170-4cab-a00b-90a1aba23223 response_code = 200 uri = /network-bootstore-config
12252023-11-20T07:52:59.510ZINFOSledAgent (RSS): Handoff to Nexus is complete file = sled-agent/src/rack_setup/service.rs:670
12262023-11-20T07:53:02.910ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::f858:c2ff:fe1d:5103": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
12272023-11-20T07:53:07.911ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::f858:c2ff:fe1d:5103": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
12282023-11-20T07:53:10.808ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:769 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:55339
12292023-11-20T07:53:10.809ZINFOSledAgent (InstanceManager): ensuring instance is registered file = sled-agent/src/instance_manager.rs:222 hardware = InstanceHardware { properties: InstanceProperties { ncpus: InstanceCpuCount(2), memory: ByteCount(1073741824), hostname: "localshark" }, nics: [NetworkInterface { id: 1a4c0397-d589-4520-83ee-1e917e933835, kind: Instance { id: 78ac0141-7ab1-4cee-9513-43da402be502 }, name: Name("net0"), ip: 172.30.0.5, mac: MacAddr(MacAddr6([168, 64, 37, 250, 203, 88])), subnet: V4(Ipv4Net(Ipv4Network { addr: 172.30.0.0, prefix: 22 })), vni: Vni(12186343), primary: true, slot: 0 }], source_nat: SourceNatConfig { ip: 10.151.2.175, first_port: 0, last_port: 16383 }, external_ips: [10.151.2.176], firewall_rules: [VpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 1a4c0397-d589-4520-83ee-1e917e933835, kind: Instance { id: 78ac0141-7ab1-4cee-9513-43da402be502 }, name: Name("net0"), ip: 172.30.0.5, mac: MacAddr(MacAddr6([168, 64, 37, 250, 203, 88])), subnet: V4(Ipv4Net(Ipv4Network { addr: 172.30.0.0, prefix: 22 })), vni: Vni(12186343), primary: true, slot: 0 }], filter_hosts: None, filter_ports: None, filter_protocols: Some([Icmp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, VpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 1a4c0397-d589-4520-83ee-1e917e933835, kind: Instance { id: 78ac0141-7ab1-4cee-9513-43da402be502 }, name: Name("net0"), ip: 172.30.0.5, mac: MacAddr(MacAddr6([168, 64, 37, 250, 203, 88])), subnet: V4(Ipv4Net(Ipv4Network { addr: 172.30.0.0, prefix: 22 })), vni: Vni(12186343), primary: true, slot: 0 }], filter_hosts: Some([Vpc(Vni(12186343))]), filter_ports: None, filter_protocols: None, action: Allow, priority: VpcFirewallRulePriority(65534) }, VpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 1a4c0397-d589-4520-83ee-1e917e933835, kind: Instance { id: 78ac0141-7ab1-4cee-9513-43da402be502 }, name: Name("net0"), ip: 172.30.0.5, mac: MacAddr(MacAddr6([168, 64, 37, 250, 203, 88])), subnet: V4(Ipv4Net(Ipv4Network { addr: 172.30.0.0, prefix: 22 })), vni: Vni(12186343), primary: true, slot: 0 }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(22), last: L4Port(22) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }], dhcp_config: DhcpConfig { dns_servers: [1.1.1.1, 9.9.9.9], host_domain: None, search_domains: [] }, disks: [DiskRequest { device: "nvme", name: "disk-9298817e87f", read_only: false, slot: Slot(0), volume_construction_request: Volume { block_size: 512, id: 4678ffc9-23a0-4490-bf13-572754357bc1, read_only_parent: Some(Url { block_size: 512, id: 47e89420-5b69-4eab-ae5c-534704f7fc14, url: "http://[fd00:1122:3344:101::1]:54321/debian-11-genericcloud-amd64.raw" }), sub_volumes: [Region { block_size: 512, blocks_per_extent: 131072, extent_count: 32, gen: 1, opts: CrucibleOpts { cert_pem: None, control: None, flush_timeout: None, id: 4678ffc9-23a0-4490-bf13-572754357bc1, key: Some("AJNRBuOH+r/u310JBhXvElq3qf/6JKoDnP8FSrv+Z4I="), key_pem: None, lossy: false, read_only: false, root_cert_pem: None, target: ["[fd00:1122:3344:101::17]:19000", "[fd00:1122:3344:101::19]:19000", "[fd00:1122:3344:101::14]:19000"] } }] } }], cloud_init_bytes: Some("6zyQTVNXSU40LjEAAgEBAAIAAioA+AEAKgBAAAAAAAAAAAAAAAApeFY0EmNpZGF0YSAgICAgRkFUMTIgICAOH75bfKwiwHQLVrQOuwcAzRBe6/Ay5M0WzRnr/lRoaXMgaXMgbm90IGEgYm9vdGFibGUgZGlzay4gIFBsZWFzZSBpbnNlcnQgYSBib290YWJsZSBmbG9wcHkgYW5kDQpwcmVzcyBhbnkga2V5IHRvIHRyeSBhZ2FpbiAuLi4gDQoAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAVar4////DwAAAAAAAAAA8P//////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////D/j///8PAAAAAAAAAADw//////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////8PY2lkYXRhICAgICAIAAAAAAAAAAAAAAAAAAAAAAAAAABBbQBlAHQAYQAtAA8A1GQAYQB0AGEAAAD//wAA/////01FVEEtRH4xICAgAABNpT50V3RXAAClPnRXAgC3AAAAQXUAcwBlAHIALQAPALBkAGEAdABhAAAA//8AAP////9VU0VSLUR+MSAgIAAATaU+dFd0VwAApT50VwAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAHsiaW5zdGFuY2UtaWQiOiI3OGFjMDE0MS03YWIxLTRjZWUtOTUxMy00M2RhNDAyYmU1MDIiLCJsb2NhbC1ob3N0bmFtZSI6ImxvY2Fsc2hhcmsiLCJwdWJsaWMta2V5cyI6WyJzc2gtZWQyNTUxOSBBQUFBQzNOemFDMWxaREkxTlRFNUFBQUFJRXBUQUZSOExZY0R1dW5nYklkelFVVXJvdmRYZFRkQmIwSUU3Y01VREFNdiJdfQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA") } instance_id = 78ac0141-7ab1-4cee-9513-43da402be502 instance_runtime = InstanceRuntimeState { propolis_id: Some(aefc4fdc-f4b3-4917-b457-6f9ddfd27caf), dst_propolis_id: None, migration_id: None, gen: Generation(3), time_updated: 2023-11-20T07:53:10.481704941Z } propolis_addr = [fd00:1122:3344:101::22]:12400 propolis_id = aefc4fdc-f4b3-4917-b457-6f9ddfd27caf vmm_runtime = VmmRuntimeState { state: Starting, gen: Generation(1), time_updated: 2023-11-20T07:53:10.431145Z }
12302023-11-20T07:53:10.809ZINFOSledAgent (InstanceManager): registering new instance file = sled-agent/src/instance_manager.rs:256 instance_id = 78ac0141-7ab1-4cee-9513-43da402be502
12312023-11-20T07:53:10.810ZINFOSledAgent (InstanceManager): initializing new Instance file = sled-agent/src/instance.rs:596 instance_id = 78ac0141-7ab1-4cee-9513-43da402be502 propolis_id = aefc4fdc-f4b3-4917-b457-6f9ddfd27caf state = InstanceInitialState { hardware: InstanceHardware { properties: InstanceProperties { ncpus: InstanceCpuCount(2), memory: ByteCount(1073741824), hostname: "localshark" }, nics: [NetworkInterface { id: 1a4c0397-d589-4520-83ee-1e917e933835, kind: Instance { id: 78ac0141-7ab1-4cee-9513-43da402be502 }, name: Name("net0"), ip: 172.30.0.5, mac: MacAddr(MacAddr6([168, 64, 37, 250, 203, 88])), subnet: V4(Ipv4Net(Ipv4Network { addr: 172.30.0.0, prefix: 22 })), vni: Vni(12186343), primary: true, slot: 0 }], source_nat: SourceNatConfig { ip: 10.151.2.175, first_port: 0, last_port: 16383 }, external_ips: [10.151.2.176], firewall_rules: [VpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 1a4c0397-d589-4520-83ee-1e917e933835, kind: Instance { id: 78ac0141-7ab1-4cee-9513-43da402be502 }, name: Name("net0"), ip: 172.30.0.5, mac: MacAddr(MacAddr6([168, 64, 37, 250, 203, 88])), subnet: V4(Ipv4Net(Ipv4Network { addr: 172.30.0.0, prefix: 22 })), vni: Vni(12186343), primary: true, slot: 0 }], filter_hosts: None, filter_ports: None, filter_protocols: Some([Icmp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, VpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 1a4c0397-d589-4520-83ee-1e917e933835, kind: Instance { id: 78ac0141-7ab1-4cee-9513-43da402be502 }, name: Name("net0"), ip: 172.30.0.5, mac: MacAddr(MacAddr6([168, 64, 37, 250, 203, 88])), subnet: V4(Ipv4Net(Ipv4Network { addr: 172.30.0.0, prefix: 22 })), vni: Vni(12186343), primary: true, slot: 0 }], filter_hosts: Some([Vpc(Vni(12186343))]), filter_ports: None, filter_protocols: None, action: Allow, priority: VpcFirewallRulePriority(65534) }, VpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 1a4c0397-d589-4520-83ee-1e917e933835, kind: Instance { id: 78ac0141-7ab1-4cee-9513-43da402be502 }, name: Name("net0"), ip: 172.30.0.5, mac: MacAddr(MacAddr6([168, 64, 37, 250, 203, 88])), subnet: V4(Ipv4Net(Ipv4Network { addr: 172.30.0.0, prefix: 22 })), vni: Vni(12186343), primary: true, slot: 0 }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(22), last: L4Port(22) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }], dhcp_config: DhcpConfig { dns_servers: [1.1.1.1, 9.9.9.9], host_domain: None, search_domains: [] }, disks: [DiskRequest { device: "nvme", name: "disk-9298817e87f", read_only: false, slot: Slot(0), volume_construction_request: Volume { block_size: 512, id: 4678ffc9-23a0-4490-bf13-572754357bc1, read_only_parent: Some(Url { block_size: 512, id: 47e89420-5b69-4eab-ae5c-534704f7fc14, url: "http://[fd00:1122:3344:101::1]:54321/debian-11-genericcloud-amd64.raw" }), sub_volumes: [Region { block_size: 512, blocks_per_extent: 131072, extent_count: 32, gen: 1, opts: CrucibleOpts { cert_pem: None, control: None, flush_timeout: None, id: 4678ffc9-23a0-4490-bf13-572754357bc1, key: Some("AJNRBuOH+r/u310JBhXvElq3qf/6JKoDnP8FSrv+Z4I="), key_pem: None, lossy: false, read_only: false, root_cert_pem: None, target: ["[fd00:1122:3344:101::17]:19000", "[fd00:1122:3344:101::19]:19000", "[fd00:1122:3344:101::14]:19000"] } }] } }], cloud_init_bytes: Some("6zyQTVNXSU40LjEAAgEBAAIAAioA+AEAKgBAAAAAAAAAAAAAAAApeFY0EmNpZGF0YSAgICAgRkFUMTIgICAOH75bfKwiwHQLVrQOuwcAzRBe6/Ay5M0WzRnr/lRoaXMgaXMgbm90IGEgYm9vdGFibGUgZGlzay4gIFBsZWFzZSBpbnNlcnQgYSBib290YWJsZSBmbG9wcHkgYW5kDQpwcmVzcyBhbnkga2V5IHRvIHRyeSBhZ2FpbiAuLi4gDQoAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAVar4////DwAAAAAAAAAA8P//////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////D/j///8PAAAAAAAAAADw//////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////8PY2lkYXRhICAgICAIAAAAAAAAAAAAAAAAAAAAAAAAAABBbQBlAHQAYQAtAA8A1GQAYQB0AGEAAAD//wAA/////01FVEEtRH4xICAgAABNpT50V3RXAAClPnRXAgC3AAAAQXUAcwBlAHIALQAPALBkAGEAdABhAAAA//8AAP////9VU0VSLUR+MSAgIAAATaU+dFd0VwAApT50VwAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAHsiaW5zdGFuY2UtaWQiOiI3OGFjMDE0MS03YWIxLTRjZWUtOTUxMy00M2RhNDAyYmU1MDIiLCJsb2NhbC1ob3N0bmFtZSI6ImxvY2Fsc2hhcmsiLCJwdWJsaWMta2V5cyI6WyJzc2gtZWQyNTUxOSBBQUFBQzNOemFDMWxaREkxTlRFNUFBQUFJRXBUQUZSOExZY0R1dW5nYklkelFVVXJvdmRYZFRkQmIwSUU3Y01VREFNdiJdfQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA") }, instance_runtime: InstanceRuntimeState { propolis_id: Some(aefc4fdc-f4b3-4917-b457-6f9ddfd27caf), dst_propolis_id: None, migration_id: None, gen: Generation(3), time_updated: 2023-11-20T07:53:10.481704941Z }, vmm_runtime: VmmRuntimeState { state: Starting, gen: Generation(1), time_updated: 2023-11-20T07:53:10.431145Z }, propolis_addr: [fd00:1122:3344:101::22]:12400 }
12322023-11-20T07:53:10.810ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 1315 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:55339 req_id = 72778059-86a0-40fe-8526-d13b8d5cea21 response_code = 200 uri = /instances/78ac0141-7ab1-4cee-9513-43da402be502
12332023-11-20T07:53:10.905ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:769 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:38131
12342023-11-20T07:53:10.925ZINFOSledAgent (PortManager): Created OPTE port file = illumos-utils/src/opte/port_manager.rs:383 port = Port { inner: PortInner { name: "opte6", _ip: 172.30.0.5, mac: MacAddr6([168, 64, 37, 250, 203, 88]), slot: 0, vni: Vni { inner: 12186343 }, gateway: Gateway { mac: MacAddr6([168, 64, 37, 255, 119, 119]), ip: 172.30.0.1 }, vnic: "vopte6" } }
12352023-11-20T07:53:10.925ZINFOSledAgent (StorageManager): Received GetLatestResources(Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) }) file = sled-storage/src/manager.rs:346
12362023-11-20T07:53:10.957ZINFOSledAgent (InstanceManager): Configuring new Omicron zone: oxz_propolis-server_aefc4fdc-f4b3-4917-b457-6f9ddfd27caf file = illumos-utils/src/zone.rs:314 instance_id = 78ac0141-7ab1-4cee-9513-43da402be502
12372023-11-20T07:53:10.978ZINFOSledAgent (InstanceManager): Installing Omicron zone: oxz_propolis-server_aefc4fdc-f4b3-4917-b457-6f9ddfd27caf file = illumos-utils/src/zone.rs:353 instance_id = 78ac0141-7ab1-4cee-9513-43da402be502
12382023-11-20T07:53:11.426ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:769 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:54529
12392023-11-20T07:53:11.427ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 117 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:54529 req_id = a4f79d5d-5081-46c9-b638-75880089f86d response_code = 200 uri = /baseboard
12402023-11-20T07:53:12.469ZINFOSledAgent (InstanceManager): Profile for oxz_propolis-server_aefc4fdc-f4b3-4917-b457-6f9ddfd27caf: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="system/illumos/propolis-server"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value="oxControlInstance0"/> <propval type="astring" name="gateway" value="fd00:1122:3344:101::1"/> <propval type="astring" name="listen_addr" value="fd00:1122:3344:101::22"/> <propval type="astring" name="listen_port" value="12400"/> <propval type="astring" name="metric_addr" value="[fd00:1122:3344:101::b]:12221"/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34 instance_id = 78ac0141-7ab1-4cee-9513-43da402be502
12412023-11-20T07:53:12.470ZINFOSledAgent (InstanceManager): Zone booting file = illumos-utils/src/running_zone.rs:509 instance_id = 78ac0141-7ab1-4cee-9513-43da402be502 zone = oxz_propolis-server_aefc4fdc-f4b3-4917-b457-6f9ddfd27caf
12422023-11-20T07:53:12.663ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_aefc4fdc-f4b3-4917-b457-6f9ddfd27caf") failed: Property not found. retry in 72.953911ms file = illumos-utils/src/svc.rs:36 instance_id = 78ac0141-7ab1-4cee-9513-43da402be502 zone = oxz_propolis-server_aefc4fdc-f4b3-4917-b457-6f9ddfd27caf
12432023-11-20T07:53:12.754ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_aefc4fdc-f4b3-4917-b457-6f9ddfd27caf") failed: Property not found. retry in 122.398146ms file = illumos-utils/src/svc.rs:36 instance_id = 78ac0141-7ab1-4cee-9513-43da402be502 zone = oxz_propolis-server_aefc4fdc-f4b3-4917-b457-6f9ddfd27caf
12442023-11-20T07:53:12.888ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_aefc4fdc-f4b3-4917-b457-6f9ddfd27caf") failed: Property not found. retry in 298.772571ms file = illumos-utils/src/svc.rs:36 instance_id = 78ac0141-7ab1-4cee-9513-43da402be502 zone = oxz_propolis-server_aefc4fdc-f4b3-4917-b457-6f9ddfd27caf
12452023-11-20T07:53:12.913ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::f858:c2ff:fe1d:5103": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
12462023-11-20T07:53:13.199ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_aefc4fdc-f4b3-4917-b457-6f9ddfd27caf") failed: Property not found. retry in 457.332981ms file = illumos-utils/src/svc.rs:36 instance_id = 78ac0141-7ab1-4cee-9513-43da402be502 zone = oxz_propolis-server_aefc4fdc-f4b3-4917-b457-6f9ddfd27caf
12472023-11-20T07:53:13.668ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_aefc4fdc-f4b3-4917-b457-6f9ddfd27caf") failed: Property not found. retry in 820.43762ms file = illumos-utils/src/svc.rs:36 instance_id = 78ac0141-7ab1-4cee-9513-43da402be502 zone = oxz_propolis-server_aefc4fdc-f4b3-4917-b457-6f9ddfd27caf
12482023-11-20T07:53:14.498ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_aefc4fdc-f4b3-4917-b457-6f9ddfd27caf") failed: Property not found. retry in 1.103582088s file = illumos-utils/src/svc.rs:36 instance_id = 78ac0141-7ab1-4cee-9513-43da402be502 zone = oxz_propolis-server_aefc4fdc-f4b3-4917-b457-6f9ddfd27caf
12492023-11-20T07:53:15.613ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_aefc4fdc-f4b3-4917-b457-6f9ddfd27caf") failed: Property not found. retry in 693.30171ms file = illumos-utils/src/svc.rs:36 instance_id = 78ac0141-7ab1-4cee-9513-43da402be502 zone = oxz_propolis-server_aefc4fdc-f4b3-4917-b457-6f9ddfd27caf
12502023-11-20T07:53:16.316ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_aefc4fdc-f4b3-4917-b457-6f9ddfd27caf") failed: Property not found. retry in 1.497225928s file = illumos-utils/src/svc.rs:36 instance_id = 78ac0141-7ab1-4cee-9513-43da402be502 zone = oxz_propolis-server_aefc4fdc-f4b3-4917-b457-6f9ddfd27caf
12512023-11-20T07:53:17.845ZINFOSledAgent (InstanceManager): Started propolis in zone: oxz_propolis-server_aefc4fdc-f4b3-4917-b457-6f9ddfd27caf file = sled-agent/src/instance.rs:984 instance_id = 78ac0141-7ab1-4cee-9513-43da402be502
12522023-11-20T07:53:17.854ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_aefc4fdc-f4b3-4917-b457-6f9ddfd27caf") failed: Property not found. retry in 69.527268ms file = illumos-utils/src/svc.rs:36 instance_id = 78ac0141-7ab1-4cee-9513-43da402be502
12532023-11-20T07:53:17.914ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::f858:c2ff:fe1d:5103": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
12542023-11-20T07:53:17.933ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_aefc4fdc-f4b3-4917-b457-6f9ddfd27caf") failed: Property not found. retry in 118.837026ms file = illumos-utils/src/svc.rs:36 instance_id = 78ac0141-7ab1-4cee-9513-43da402be502
12552023-11-20T07:53:18.062ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_aefc4fdc-f4b3-4917-b457-6f9ddfd27caf") failed: Property not found. retry in 262.494862ms file = illumos-utils/src/svc.rs:36 instance_id = 78ac0141-7ab1-4cee-9513-43da402be502
12562023-11-20T07:53:18.336ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_aefc4fdc-f4b3-4917-b457-6f9ddfd27caf") failed: Property not found. retry in 239.366602ms file = illumos-utils/src/svc.rs:36 instance_id = 78ac0141-7ab1-4cee-9513-43da402be502
12572023-11-20T07:53:18.587ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_aefc4fdc-f4b3-4917-b457-6f9ddfd27caf") failed: Property not found. retry in 479.517112ms file = illumos-utils/src/svc.rs:36 instance_id = 78ac0141-7ab1-4cee-9513-43da402be502
12582023-11-20T07:53:19.077ZINFOSledAgent (InstanceManager): Propolis SMF service is online file = sled-agent/src/instance.rs:993 instance_id = 78ac0141-7ab1-4cee-9513-43da402be502
12592023-11-20T07:53:19.112ZINFOSledAgent (InstanceManager): Propolis HTTP server online file = sled-agent/src/instance.rs:1007 instance_id = 78ac0141-7ab1-4cee-9513-43da402be502
12602023-11-20T07:53:19.113ZINFOSledAgent (InstanceManager): Sending ensure request to propolis: InstanceEnsureRequest { cloud_init_bytes: Some("6zyQTVNXSU40LjEAAgEBAAIAAioA+AEAKgBAAAAAAAAAAAAAAAApeFY0EmNpZGF0YSAgICAgRkFUMTIgICAOH75bfKwiwHQLVrQOuwcAzRBe6/Ay5M0WzRnr/lRoaXMgaXMgbm90IGEgYm9vdGFibGUgZGlzay4gIFBsZWFzZSBpbnNlcnQgYSBib290YWJsZSBmbG9wcHkgYW5kDQpwcmVzcyBhbnkga2V5IHRvIHRyeSBhZ2FpbiAuLi4gDQoAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAVar4////DwAAAAAAAAAA8P//////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////D/j///8PAAAAAAAAAADw//////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////8PY2lkYXRhICAgICAIAAAAAAAAAAAAAAAAAAAAAAAAAABBbQBlAHQAYQAtAA8A1GQAYQB0AGEAAAD//wAA/////01FVEEtRH4xICAgAABNpT50V3RXAAClPnRXAgC3AAAAQXUAcwBlAHIALQAPALBkAGEAdABhAAAA//8AAP////9VU0VSLUR+MSAgIAAATaU+dFd0VwAApT50VwAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAHsiaW5zdGFuY2UtaWQiOiI3OGFjMDE0MS03YWIxLTRjZWUtOTUxMy00M2RhNDAyYmU1MDIiLCJsb2NhbC1ob3N0bmFtZSI6ImxvY2Fsc2hhcmsiLCJwdWJsaWMta2V5cyI6WyJzc2gtZWQyNTUxOSBBQUFBQzNOemFDMWxaREkxTlRFNUFBQUFJRXBUQUZSOExZY0R1dW5nYklkelFVVXJvdmRYZFRkQmIwSUU3Y01VREFNdiJdfQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA"), disks: [DiskRequest { device: "nvme", name: "disk-9298817e87f", read_only: false, slot: Slot(0), volume_construction_request: Volume { block_size: 512, id: 4678ffc9-23a0-4490-bf13-572754357bc1, read_only_parent: Some(Url { block_size: 512, id: 47e89420-5b69-4eab-ae5c-534704f7fc14, url: "http://[fd00:1122:3344:101::1]:54321/debian-11-genericcloud-amd64.raw" }), sub_volumes: [Region { block_size: 512, blocks_per_extent: 131072, extent_count: 32, gen: 1, opts: CrucibleOpts { cert_pem: None, control: None, flush_timeout: None, id: 4678ffc9-23a0-4490-bf13-572754357bc1, key: Some("AJNRBuOH+r/u310JBhXvElq3qf/6JKoDnP8FSrv+Z4I="), key_pem: None, lossy: false, read_only: false, root_cert_pem: None, target: ["[fd00:1122:3344:101::17]:19000", "[fd00:1122:3344:101::19]:19000", "[fd00:1122:3344:101::14]:19000"] } }] } }], migrate: None, nics: [NetworkInterfaceRequest { name: "vopte6", slot: Slot(0) }], properties: InstanceProperties { bootrom_id: 00000000-0000-0000-0000-000000000000, description: "Test description", id: 78ac0141-7ab1-4cee-9513-43da402be502, image_id: 00000000-0000-0000-0000-000000000000, memory: 1024, name: "localshark", vcpus: 2 } } file = sled-agent/src/instance.rs:447 instance_id = 78ac0141-7ab1-4cee-9513-43da402be502
12612023-11-20T07:53:19.305ZINFOSledAgent (InstanceManager): result of instance_ensure call is Ok(InstanceEnsureResponse { migrate: None }) file = sled-agent/src/instance.rs:449 instance_id = 78ac0141-7ab1-4cee-9513-43da402be502
12622023-11-20T07:53:19.305ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 8400439 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:38131 req_id = 2bd82ff3-b197-49ff-a9e3-0b36f9bb8f94 response_code = 200 uri = /instances/78ac0141-7ab1-4cee-9513-43da402be502/state
12632023-11-20T07:53:19.306ZINFOSledAgent (InstanceManager): Observing new propolis state: ObservedPropolisState { vmm_state: PropolisInstanceState(Starting), migration_status: NoMigration, time: 2023-11-20T07:53:19.306065274Z } file = sled-agent/src/instance.rs:331 instance_id = 78ac0141-7ab1-4cee-9513-43da402be502
12642023-11-20T07:53:19.306ZINFOSledAgent (InstanceManager): updated state after observing Propolis state change file = sled-agent/src/instance.rs:353 instance_id = 78ac0141-7ab1-4cee-9513-43da402be502 new_instance_state = InstanceRuntimeState { propolis_id: Some(aefc4fdc-f4b3-4917-b457-6f9ddfd27caf), dst_propolis_id: None, migration_id: None, gen: Generation(3), time_updated: 2023-11-20T07:53:10.481704941Z } new_vmm_state = VmmRuntimeState { state: Starting, gen: Generation(2), time_updated: 2023-11-20T07:53:19.306065274Z } propolis_id = aefc4fdc-f4b3-4917-b457-6f9ddfd27caf
12652023-11-20T07:53:19.306ZINFOSledAgent (InstanceManager): Publishing instance state update to Nexus file = sled-agent/src/instance.rs:256 instance_id = 78ac0141-7ab1-4cee-9513-43da402be502 state = SledInstanceState { instance_state: InstanceRuntimeState { propolis_id: Some(aefc4fdc-f4b3-4917-b457-6f9ddfd27caf), dst_propolis_id: None, migration_id: None, gen: Generation(3), time_updated: 2023-11-20T07:53:10.481704941Z }, propolis_id: aefc4fdc-f4b3-4917-b457-6f9ddfd27caf, vmm_state: VmmRuntimeState { state: Starting, gen: Generation(2), time_updated: 2023-11-20T07:53:19.306065274Z } }
12662023-11-20T07:53:19.343ZINFOSledAgent (InstanceManager): Observing new propolis state: ObservedPropolisState { vmm_state: PropolisInstanceState(Running), migration_status: NoMigration, time: 2023-11-20T07:53:19.343027713Z } file = sled-agent/src/instance.rs:331 instance_id = 78ac0141-7ab1-4cee-9513-43da402be502
12672023-11-20T07:53:19.343ZINFOSledAgent (InstanceManager): updated state after observing Propolis state change file = sled-agent/src/instance.rs:353 instance_id = 78ac0141-7ab1-4cee-9513-43da402be502 new_instance_state = InstanceRuntimeState { propolis_id: Some(aefc4fdc-f4b3-4917-b457-6f9ddfd27caf), dst_propolis_id: None, migration_id: None, gen: Generation(3), time_updated: 2023-11-20T07:53:10.481704941Z } new_vmm_state = VmmRuntimeState { state: Running, gen: Generation(3), time_updated: 2023-11-20T07:53:19.343027713Z } propolis_id = aefc4fdc-f4b3-4917-b457-6f9ddfd27caf
12682023-11-20T07:53:19.343ZINFOSledAgent (InstanceManager): Publishing instance state update to Nexus file = sled-agent/src/instance.rs:256 instance_id = 78ac0141-7ab1-4cee-9513-43da402be502 state = SledInstanceState { instance_state: InstanceRuntimeState { propolis_id: Some(aefc4fdc-f4b3-4917-b457-6f9ddfd27caf), dst_propolis_id: None, migration_id: None, gen: Generation(3), time_updated: 2023-11-20T07:53:10.481704941Z }, propolis_id: aefc4fdc-f4b3-4917-b457-6f9ddfd27caf, vmm_state: VmmRuntimeState { state: Running, gen: Generation(3), time_updated: 2023-11-20T07:53:19.343027713Z } }
12692023-11-20T07:53:21.078ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:769 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::d]:35355
12702023-11-20T07:53:21.079ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 1286 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::d]:35355 req_id = 292a85db-4874-4ae9-8ed6-aec473cdafed response_code = 200 uri = /metrics/collect/e75545d5-2741-43ba-8c56-e7f2943a5b82
12712023-11-20T07:53:22.917ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::f858:c2ff:fe1d:5103": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
12722023-11-20T07:53:27.918ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::f858:c2ff:fe1d:5103": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
12732023-11-20T07:53:32.919ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::f858:c2ff:fe1d:5103": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
12742023-11-20T07:53:37.919ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::f858:c2ff:fe1d:5103": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
12752023-11-20T07:53:41.459ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:769 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:53894
12762023-11-20T07:53:41.459ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 36 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:53894 req_id = 3a8eb97b-9153-4f3d-9ad2-16ea4a117026 response_code = 200 uri = /baseboard
12772023-11-20T07:53:42.921ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::f858:c2ff:fe1d:5103": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
12782023-11-20T07:53:47.922ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::f858:c2ff:fe1d:5103": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
12792023-11-20T07:53:51.078ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 312 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::d]:35355 req_id = ea0eb681-5629-4b52-8b9a-080a4c0e481c response_code = 200 uri = /metrics/collect/e75545d5-2741-43ba-8c56-e7f2943a5b82
12802023-11-20T07:53:52.924ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::f858:c2ff:fe1d:5103": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
12812023-11-20T07:53:57.925ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::f858:c2ff:fe1d:5103": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
12822023-11-20T07:54:02.926ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::f858:c2ff:fe1d:5103": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
12832023-11-20T07:54:07.928ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::f858:c2ff:fe1d:5103": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
12842023-11-20T07:54:11.491ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:769 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:47458
12852023-11-20T07:54:11.492ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 754 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:47458 req_id = 293f153c-2d51-48ef-84cd-48186f74be9d response_code = 200 uri = /baseboard
12862023-11-20T07:54:12.929ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::f858:c2ff:fe1d:5103": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
12872023-11-20T07:54:17.931ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::f858:c2ff:fe1d:5103": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
12882023-11-20T07:54:21.078ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 222 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::d]:35355 req_id = c4e38282-2852-42d9-93da-bf4944b67627 response_code = 200 uri = /metrics/collect/e75545d5-2741-43ba-8c56-e7f2943a5b82
12892023-11-20T07:54:22.932ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::f858:c2ff:fe1d:5103": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
12902023-11-20T07:54:27.934ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::f858:c2ff:fe1d:5103": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
12912023-11-20T07:54:32.935ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::f858:c2ff:fe1d:5103": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
12922023-11-20T07:54:37.938ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::f858:c2ff:fe1d:5103": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
12932023-11-20T07:54:41.524ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:769 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:33203
12942023-11-20T07:54:41.524ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 31 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:33203 req_id = 90ea9aef-c05d-4df4-bebc-fe86fe098c05 response_code = 200 uri = /baseboard
12952023-11-20T07:54:42.940ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::f858:c2ff:fe1d:5103": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
12962023-11-20T07:54:47.941ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::f858:c2ff:fe1d:5103": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
12972023-11-20T07:54:51.077ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 312 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::d]:35355 req_id = 1695765e-c93a-4b3d-836e-5a82d5115642 response_code = 200 uri = /metrics/collect/e75545d5-2741-43ba-8c56-e7f2943a5b82
12982023-11-20T07:54:52.942ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::f858:c2ff:fe1d:5103": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
12992023-11-20T07:54:57.943ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::f858:c2ff:fe1d:5103": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
13002023-11-20T07:55:02.945ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::f858:c2ff:fe1d:5103": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
13012023-11-20T07:55:07.947ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::f858:c2ff:fe1d:5103": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
13022023-11-20T07:55:11.557ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:769 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:38808
13032023-11-20T07:55:11.557ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 48 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:38808 req_id = 91716e1b-63a2-41fc-ae48-ee69ecacd674 response_code = 200 uri = /baseboard
13042023-11-20T07:55:11.914ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:532
13052023-11-20T07:55:12.948ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::f858:c2ff:fe1d:5103": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
13062023-11-20T07:55:17.949ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::f858:c2ff:fe1d:5103": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
13072023-11-20T07:55:21.077ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 259 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::d]:35355 req_id = 880a038d-7092-4aaf-8c8a-f1fa087789d8 response_code = 200 uri = /metrics/collect/e75545d5-2741-43ba-8c56-e7f2943a5b82
13082023-11-20T07:55:22.951ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::f858:c2ff:fe1d:5103": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
13092023-11-20T07:55:27.953ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::f858:c2ff:fe1d:5103": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
13102023-11-20T07:55:32.955ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::f858:c2ff:fe1d:5103": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
13112023-11-20T07:55:37.958ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::f858:c2ff:fe1d:5103": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
13122023-11-20T07:55:41.591ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:769 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:41601
13132023-11-20T07:55:41.591ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 34 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:41601 req_id = b927e201-52d9-4ad1-bc24-c4fbf7e5b262 response_code = 200 uri = /baseboard
13142023-11-20T07:55:42.959ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::f858:c2ff:fe1d:5103": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
13152023-11-20T07:55:47.961ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::f858:c2ff:fe1d:5103": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
13162023-11-20T07:55:51.078ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 362 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::d]:35355 req_id = 06c0159b-22a5-44ae-b114-e83482f0b56c response_code = 200 uri = /metrics/collect/e75545d5-2741-43ba-8c56-e7f2943a5b82
13172023-11-20T07:55:52.963ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::f858:c2ff:fe1d:5103": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
13182023-11-20T07:55:57.965ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::f858:c2ff:fe1d:5103": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
13192023-11-20T07:56:01.460ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:769 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:35194
13202023-11-20T07:56:01.461ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 498 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:35194 req_id = 6aa00929-7755-4a84-919e-212c1779ca57 response_code = 200 uri = /instances/78ac0141-7ab1-4cee-9513-43da402be502/state
13212023-11-20T07:56:01.461ZINFOSledAgent (InstanceManager): Observing new propolis state: ObservedPropolisState { vmm_state: PropolisInstanceState(Stopping), migration_status: NoMigration, time: 2023-11-20T07:56:01.461420978Z } file = sled-agent/src/instance.rs:331 instance_id = 78ac0141-7ab1-4cee-9513-43da402be502
13222023-11-20T07:56:01.461ZINFOSledAgent (InstanceManager): updated state after observing Propolis state change file = sled-agent/src/instance.rs:353 instance_id = 78ac0141-7ab1-4cee-9513-43da402be502 new_instance_state = InstanceRuntimeState { propolis_id: Some(aefc4fdc-f4b3-4917-b457-6f9ddfd27caf), dst_propolis_id: None, migration_id: None, gen: Generation(3), time_updated: 2023-11-20T07:53:10.481704941Z } new_vmm_state = VmmRuntimeState { state: Stopping, gen: Generation(5), time_updated: 2023-11-20T07:56:01.461420978Z } propolis_id = aefc4fdc-f4b3-4917-b457-6f9ddfd27caf
13232023-11-20T07:56:01.461ZINFOSledAgent (InstanceManager): Publishing instance state update to Nexus file = sled-agent/src/instance.rs:256 instance_id = 78ac0141-7ab1-4cee-9513-43da402be502 state = SledInstanceState { instance_state: InstanceRuntimeState { propolis_id: Some(aefc4fdc-f4b3-4917-b457-6f9ddfd27caf), dst_propolis_id: None, migration_id: None, gen: Generation(3), time_updated: 2023-11-20T07:53:10.481704941Z }, propolis_id: aefc4fdc-f4b3-4917-b457-6f9ddfd27caf, vmm_state: VmmRuntimeState { state: Stopping, gen: Generation(5), time_updated: 2023-11-20T07:56:01.461420978Z } }
13242023-11-20T07:56:01.511ZINFOSledAgent (InstanceManager): Observing new propolis state: ObservedPropolisState { vmm_state: PropolisInstanceState(Stopped), migration_status: NoMigration, time: 2023-11-20T07:56:01.511025873Z } file = sled-agent/src/instance.rs:331 instance_id = 78ac0141-7ab1-4cee-9513-43da402be502
13252023-11-20T07:56:01.511ZINFOSledAgent (InstanceManager): updated state after observing Propolis state change file = sled-agent/src/instance.rs:353 instance_id = 78ac0141-7ab1-4cee-9513-43da402be502 new_instance_state = InstanceRuntimeState { propolis_id: Some(aefc4fdc-f4b3-4917-b457-6f9ddfd27caf), dst_propolis_id: None, migration_id: None, gen: Generation(3), time_updated: 2023-11-20T07:53:10.481704941Z } new_vmm_state = VmmRuntimeState { state: Stopping, gen: Generation(6), time_updated: 2023-11-20T07:56:01.511025873Z } propolis_id = aefc4fdc-f4b3-4917-b457-6f9ddfd27caf
13262023-11-20T07:56:01.511ZINFOSledAgent (InstanceManager): Publishing instance state update to Nexus file = sled-agent/src/instance.rs:256 instance_id = 78ac0141-7ab1-4cee-9513-43da402be502 state = SledInstanceState { instance_state: InstanceRuntimeState { propolis_id: Some(aefc4fdc-f4b3-4917-b457-6f9ddfd27caf), dst_propolis_id: None, migration_id: None, gen: Generation(3), time_updated: 2023-11-20T07:53:10.481704941Z }, propolis_id: aefc4fdc-f4b3-4917-b457-6f9ddfd27caf, vmm_state: VmmRuntimeState { state: Stopping, gen: Generation(6), time_updated: 2023-11-20T07:56:01.511025873Z } }
13272023-11-20T07:56:01.647ZINFOSledAgent (InstanceManager): Observing new propolis state: ObservedPropolisState { vmm_state: PropolisInstanceState(Destroyed), migration_status: NoMigration, time: 2023-11-20T07:56:01.647802285Z } file = sled-agent/src/instance.rs:331 instance_id = 78ac0141-7ab1-4cee-9513-43da402be502
13282023-11-20T07:56:01.647ZINFOSledAgent (InstanceManager): updated state after observing Propolis state change file = sled-agent/src/instance.rs:353 instance_id = 78ac0141-7ab1-4cee-9513-43da402be502 new_instance_state = InstanceRuntimeState { propolis_id: None, dst_propolis_id: None, migration_id: None, gen: Generation(5), time_updated: 2023-11-20T07:56:01.647802285Z } new_vmm_state = VmmRuntimeState { state: Destroyed, gen: Generation(7), time_updated: 2023-11-20T07:56:01.647802285Z } propolis_id = aefc4fdc-f4b3-4917-b457-6f9ddfd27caf
13292023-11-20T07:56:01.647ZINFOSledAgent (InstanceManager): terminating VMM that has exited file = sled-agent/src/instance.rs:368 instance_id = 78ac0141-7ab1-4cee-9513-43da402be502
13302023-11-20T07:56:01.647ZINFOSledAgent (StorageManager): Received GetLatestResources(Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) }) file = sled-storage/src/manager.rs:346
13312023-11-20T07:56:01.649ZINFOSledAgent (StorageManager): Received GetLatestResources(Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) }) file = sled-storage/src/manager.rs:346
13322023-11-20T07:56:01.649ZINFOSledAgent (ZoneBundler): creating zone bundle context = ZoneBundleContext { storage_dirs: ["/pool/int/a462a7f7-b628-40fe-80ff-4e4189e2d62b/debug/bundle/zone", "/pool/int/b462a7f7-b628-40fe-80ff-4e4189e2d62b/debug/bundle/zone"], cause: TerminatedInstance, extra_log_dirs: ["/pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/debug", "/pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/debug", "/pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/debug", "/pool/ext/616b26df-e62a-4c68-b506-f4a923d8aaf7/crypt/debug", "/pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/debug", "/pool/ext/ceb4461c-cf56-4719-ad3c-14430bfdfb60/crypt/debug", "/pool/ext/d462a7f7-b628-40fe-80ff-4e4189e2d62b/crypt/debug", "/pool/ext/e4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/debug", "/pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/debug"] } file = sled-agent/src/zone_bundle.rs:436 zone_name = oxz_propolis-server_aefc4fdc-f4b3-4917-b457-6f9ddfd27caf
13332023-11-20T07:56:02.293ZINFOSledAgent (ZoneBundler): finished zone bundle file = sled-agent/src/zone_bundle.rs:1229 metadata = ZoneBundleMetadata { id: ZoneBundleId { zone_name: "oxz_propolis-server_aefc4fdc-f4b3-4917-b457-6f9ddfd27caf", bundle_id: 2b0ad3e0-0947-4449-bb23-9879af104686 }, time_created: 2023-11-20T07:56:01.649850365Z, version: 0, cause: TerminatedInstance }
13342023-11-20T07:56:02.293ZWARNSledAgent (InstanceManager): Halting and removing zone: oxz_propolis-server_aefc4fdc-f4b3-4917-b457-6f9ddfd27caf file = sled-agent/src/instance.rs:541 instance_id = 78ac0141-7ab1-4cee-9513-43da402be502
13352023-11-20T07:56:02.967ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::f858:c2ff:fe1d:5103": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
13362023-11-20T07:56:03.856ZINFOSledAgent (InstanceManager): halt_and_remove_logged: Previous zone state: Running file = illumos-utils/src/zone.rs:268 instance_id = 78ac0141-7ab1-4cee-9513-43da402be502
13372023-11-20T07:56:03.885ZINFOSledAgent (InstanceManager): Publishing instance state update to Nexus file = sled-agent/src/instance.rs:256 instance_id = 78ac0141-7ab1-4cee-9513-43da402be502 state = SledInstanceState { instance_state: InstanceRuntimeState { propolis_id: None, dst_propolis_id: None, migration_id: None, gen: Generation(5), time_updated: 2023-11-20T07:56:01.647802285Z }, propolis_id: aefc4fdc-f4b3-4917-b457-6f9ddfd27caf, vmm_state: VmmRuntimeState { state: Destroyed, gen: Generation(7), time_updated: 2023-11-20T07:56:01.647802285Z } }
13382023-11-20T07:56:03.995ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:769 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:57448
13392023-11-20T07:56:03.995ZWARNSledAgent (PortManager): unset_virtual_nic_host unimplmented file = illumos-utils/src/opte/port_manager.rs:494
13402023-11-20T07:56:03.995ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 208 local_addr = [fd00:1122:3344:101::1]:12345 method = DELETE remote_addr = [fd00:1122:3344:101::c]:57448 req_id = b2a9f6a1-05f4-432e-948f-d8b77b0525d8 response_code = 204 uri = /v2p/1a4c0397-d589-4520-83ee-1e917e933835
13412023-11-20T07:56:04.095ZINFOSledAgent (InstanceManager): State monitoring task complete file = sled-agent/src/instance.rs:484 instance_id = 78ac0141-7ab1-4cee-9513-43da402be502