Max level shown:
1 [ Dec 6 17:40:38 Enabled. ]
2 [ Dec 6 17:40:38 Executing start method ("ctrun -l child -o noorphan,regent /opt/oxide/sled-agent/sled-agent run /opt/oxide/sled-agent/pkg/config.toml &"). ]
3 [ Dec 6 17:40:38 Method "start" exited with status 0. ]
4 [ Dec 6 17:40:38 Rereading configuration. ]
5 [ Dec 6 17:40:38 No 'refresh' method defined. Treating as :true. ]
6 note: configured to log to "/dev/stdout"
72023-12-06T17:40:38.049ZINFOSledAgent (BootstrapAgentStartup): finding links ["net0", "net1"] file = sled-agent/src/bootstrap/pre_server.rs:240
82023-12-06T17:40:38.103ZINFOSledAgent (BootstrapAgentStartup): Starting mg-ddm service addr-objs = [AddrObject { interface: "net0", name: "ll" }, AddrObject { interface: "net1", name: "ll" }] file = sled-agent/src/bootstrap/pre_server.rs:247
92023-12-06T17:40:38.103ZINFOSledAgent (BootstrapAgentStartup): Importing mg-ddm service file = sled-agent/src/bootstrap/maghemite.rs:42 path = /opt/oxide/mg-ddm/pkg/ddm/manifest.xml
102023-12-06T17:40:38.214ZINFOSledAgent (BootstrapAgentStartup): Setting mg-ddm interfaces file = sled-agent/src/bootstrap/maghemite.rs:50 interfaces = ("net0/ll" "net1/ll")
112023-12-06T17:40:38.223ZINFOSledAgent (BootstrapAgentStartup): Enabling mg-ddm service file = sled-agent/src/bootstrap/maghemite.rs:56
122023-12-06T17:40:38.231ZINFOSledAgent (BootstrapAgentStartup): Ensuring zfs key directory exists file = sled-agent/src/bootstrap/pre_server.rs:257 path = /var/run/oxide/
132023-12-06T17:40:38.836ZINFOSledAgent (BootstrapAgentStartup): Sending prefix to ddmd for advertisement DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:91 prefix = Ipv6Prefix { addr: fdb0:18c0:4d0d:a02a::, len: 64 }
142023-12-06T17:40:38.976ZINFOSledAgent: Starting KeyManager file = sled-agent/src/long_running_tasks.rs:117
152023-12-06T17:40:38.976ZINFOSledAgent: Starting StorageManager file = sled-agent/src/long_running_tasks.rs:129
162023-12-06T17:40:38.976ZINFOSledAgent: Starting StorageMonitor file = sled-agent/src/long_running_tasks.rs:141
172023-12-06T17:40:38.976ZINFOSledAgent (DumpSetup-archival): DumpSetup poll loop started. file = sled-agent/src/dump_setup.rs:171
182023-12-06T17:40:38.976ZINFOSledAgent: Starting HardwareManager file = sled-agent/src/long_running_tasks.rs:162 sled_mode = Scrimlet { asic: SoftNpuZone }
192023-12-06T17:40:38.976ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:532
202023-12-06T17:40:38.976ZINFOSledAgent (HardwareManager): Creating HardwareManager file = sled-hardware/src/illumos/mod.rs:606
212023-12-06T17:40:38.979ZINFOSledAgent (HardwareManager): Generated i86pc baseboard Pc { identifier: "c6eef3d4b5d9454d80e3983e3692cc58", model: "i86pc" } file = sled-hardware/src/illumos/mod.rs:532
222023-12-06T17:40:38.979ZWARNSledAgent (HardwareManager): Device is not a Gimlet (i86pc), proceeding with null hardware view file = sled-hardware/src/illumos/mod.rs:655
232023-12-06T17:40:38.979ZINFOSledAgent: Starting HardwareMonitor file = sled-agent/src/long_running_tasks.rs:176
242023-12-06T17:40:38.979ZINFOSledAgent (HardwareManager): Monitoring for hardware updates file = sled-hardware/src/illumos/mod.rs:713
252023-12-06T17:40:38.979ZINFOSledAgent: Upserting synthetic zpool to Storage Manager: oxi_a462a7f7-b628-40fe-80ff-4e4189e2d62b file = sled-agent/src/long_running_tasks.rs:232
262023-12-06T17:40:38.979ZINFOSledAgent: Upserting synthetic zpool to Storage Manager: oxi_b462a7f7-b628-40fe-80ff-4e4189e2d62b file = sled-agent/src/long_running_tasks.rs:232
272023-12-06T17:40:38.979ZINFOSledAgent: Upserting synthetic zpool to Storage Manager: oxp_d462a7f7-b628-40fe-80ff-4e4189e2d62b file = sled-agent/src/long_running_tasks.rs:232
282023-12-06T17:40:38.979ZINFOSledAgent: Upserting synthetic zpool to Storage Manager: oxp_e4b4dc87-ab46-49fb-a4b4-d361ae214c03 file = sled-agent/src/long_running_tasks.rs:232
292023-12-06T17:40:38.979ZINFOSledAgent: Upserting synthetic zpool to Storage Manager: oxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03 file = sled-agent/src/long_running_tasks.rs:232
302023-12-06T17:40:38.979ZINFOSledAgent: Upserting synthetic zpool to Storage Manager: oxp_14b4dc87-ab46-49fb-a4b4-d361ae214c03 file = sled-agent/src/long_running_tasks.rs:232
312023-12-06T17:40:38.979ZINFOSledAgent: Upserting synthetic zpool to Storage Manager: oxp_24b4dc87-ab46-49fb-a4b4-d361ae214c03 file = sled-agent/src/long_running_tasks.rs:232
322023-12-06T17:40:38.979ZINFOSledAgent: Upserting synthetic zpool to Storage Manager: oxp_cd70d7f6-2354-4bf2-8012-55bf9eaf7930 file = sled-agent/src/long_running_tasks.rs:232
332023-12-06T17:40:38.979ZINFOSledAgent: Upserting synthetic zpool to Storage Manager: oxp_ceb4461c-cf56-4719-ad3c-14430bfdfb60 file = sled-agent/src/long_running_tasks.rs:232
342023-12-06T17:40:38.979ZINFOSledAgent: Upserting synthetic zpool to Storage Manager: oxp_31bd71cd-4736-4a12-a387-9b74b050396f file = sled-agent/src/long_running_tasks.rs:232
352023-12-06T17:40:38.979ZINFOSledAgent: Upserting synthetic zpool to Storage Manager: oxp_616b26df-e62a-4c68-b506-f4a923d8aaf7 file = sled-agent/src/long_running_tasks.rs:232
362023-12-06T17:40:38.979ZINFOSledAgent: Waiting for boot disk file = sled-agent/src/long_running_tasks.rs:88
372023-12-06T17:40:38.979ZINFOSledAgent (HardwareMonitor): Checking current full hardware snapshot file = sled-agent/src/hardware_monitor.rs:241 underlay_network_info = None
382023-12-06T17:40:38.979ZINFOSledAgent (StorageManager): Received AddDisk(Synthetic(SyntheticDisk { identity: DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-a462a7f7-b628-40fe-80ff-4e4189e2d62b", model: "synthetic-model" }, zpool_name: ZpoolName { id: a462a7f7-b628-40fe-80ff-4e4189e2d62b, kind: Internal } })) file = sled-storage/src/manager.rs:346
392023-12-06T17:40:39.163ZINFOSledAgent (StorageManager): Channel capacity decreased current = 245 file = sled-storage/src/manager.rs:326 previous = 256
402023-12-06T17:40:39.165ZINFOSledAgent: Found boot disk DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-a462a7f7-b628-40fe-80ff-4e4189e2d62b", model: "synthetic-model" } file = sled-agent/src/long_running_tasks.rs:90
412023-12-06T17:40:39.165ZINFOSledAgent (StorageManager): Received AddDisk(Synthetic(SyntheticDisk { identity: DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-b462a7f7-b628-40fe-80ff-4e4189e2d62b", model: "synthetic-model" }, zpool_name: ZpoolName { id: b462a7f7-b628-40fe-80ff-4e4189e2d62b, kind: Internal } })) file = sled-storage/src/manager.rs:346
422023-12-06T17:40:39.331ZINFOSledAgent (StorageManager): Received AddDisk(Synthetic(SyntheticDisk { identity: DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-d462a7f7-b628-40fe-80ff-4e4189e2d62b", model: "synthetic-model" }, zpool_name: ZpoolName { id: d462a7f7-b628-40fe-80ff-4e4189e2d62b, kind: External } })) file = sled-storage/src/manager.rs:346
432023-12-06T17:40:39.331ZINFOSledAgent (StorageManager): Received AddDisk(Synthetic(SyntheticDisk { identity: DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-e4b4dc87-ab46-49fb-a4b4-d361ae214c03", model: "synthetic-model" }, zpool_name: ZpoolName { id: e4b4dc87-ab46-49fb-a4b4-d361ae214c03, kind: External } })) file = sled-storage/src/manager.rs:346
442023-12-06T17:40:39.331ZINFOSledAgent (StorageManager): Received AddDisk(Synthetic(SyntheticDisk { identity: DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-f4b4dc87-ab46-49fb-a4b4-d361ae214c03", model: "synthetic-model" }, zpool_name: ZpoolName { id: f4b4dc87-ab46-49fb-a4b4-d361ae214c03, kind: External } })) file = sled-storage/src/manager.rs:346
452023-12-06T17:40:39.331ZINFOSledAgent (StorageManager): Received AddDisk(Synthetic(SyntheticDisk { identity: DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-14b4dc87-ab46-49fb-a4b4-d361ae214c03", model: "synthetic-model" }, zpool_name: ZpoolName { id: 14b4dc87-ab46-49fb-a4b4-d361ae214c03, kind: External } })) file = sled-storage/src/manager.rs:346
462023-12-06T17:40:39.331ZINFOSledAgent (StorageManager): Received AddDisk(Synthetic(SyntheticDisk { identity: DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-24b4dc87-ab46-49fb-a4b4-d361ae214c03", model: "synthetic-model" }, zpool_name: ZpoolName { id: 24b4dc87-ab46-49fb-a4b4-d361ae214c03, kind: External } })) file = sled-storage/src/manager.rs:346
472023-12-06T17:40:39.331ZINFOSledAgent (StorageManager): Received AddDisk(Synthetic(SyntheticDisk { identity: DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-cd70d7f6-2354-4bf2-8012-55bf9eaf7930", model: "synthetic-model" }, zpool_name: ZpoolName { id: cd70d7f6-2354-4bf2-8012-55bf9eaf7930, kind: External } })) file = sled-storage/src/manager.rs:346
482023-12-06T17:40:39.331ZINFOSledAgent (StorageManager): Received AddDisk(Synthetic(SyntheticDisk { identity: DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-ceb4461c-cf56-4719-ad3c-14430bfdfb60", model: "synthetic-model" }, zpool_name: ZpoolName { id: ceb4461c-cf56-4719-ad3c-14430bfdfb60, kind: External } })) file = sled-storage/src/manager.rs:346
492023-12-06T17:40:39.331ZINFOSledAgent (StorageManager): Received AddDisk(Synthetic(SyntheticDisk { identity: DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-31bd71cd-4736-4a12-a387-9b74b050396f", model: "synthetic-model" }, zpool_name: ZpoolName { id: 31bd71cd-4736-4a12-a387-9b74b050396f, kind: External } })) file = sled-storage/src/manager.rs:346
502023-12-06T17:40:39.331ZINFOSledAgent (StorageManager): Received AddDisk(Synthetic(SyntheticDisk { identity: DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-616b26df-e62a-4c68-b506-f4a923d8aaf7", model: "synthetic-model" }, zpool_name: ZpoolName { id: 616b26df-e62a-4c68-b506-f4a923d8aaf7, kind: External } })) file = sled-storage/src/manager.rs:346
512023-12-06T17:40:39.331ZINFOSledAgent (StorageManager): Received DisksChanged({}) file = sled-storage/src/manager.rs:346
522023-12-06T17:40:39.331ZINFOSledAgent (StorageManager): Channel capacity increased current = 255 file = sled-storage/src/manager.rs:336 previous = 245
532023-12-06T17:40:39.331ZINFOSledAgent (StorageManager): Received GetLatestResources(Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) }) file = sled-storage/src/manager.rs:346
542023-12-06T17:40:39.331ZINFOSledAgent (StorageMonitor): Received storage manager update file = sled-agent/src/storage_monitor.rs:111 resources = StorageResources { disks: {DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-a462a7f7-b628-40fe-80ff-4e4189e2d62b", model: "synthetic-model" }: (Synthetic(SyntheticDisk { identity: DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-a462a7f7-b628-40fe-80ff-4e4189e2d62b", model: "synthetic-model" }, zpool_name: ZpoolName { id: a462a7f7-b628-40fe-80ff-4e4189e2d62b, kind: Internal } }), Pool { name: ZpoolName { id: a462a7f7-b628-40fe-80ff-4e4189e2d62b, kind: Internal }, info: ZpoolInfo { name: "oxi_a462a7f7-b628-40fe-80ff-4e4189e2d62b", size: 20937965568, allocated: 494592, free: 20937470976, health: Online }, parent: DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-a462a7f7-b628-40fe-80ff-4e4189e2d62b", model: "synthetic-model" } }), DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-b462a7f7-b628-40fe-80ff-4e4189e2d62b", model: "synthetic-model" }: (Synthetic(SyntheticDisk { identity: DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-b462a7f7-b628-40fe-80ff-4e4189e2d62b", model: "synthetic-model" }, zpool_name: ZpoolName { id: b462a7f7-b628-40fe-80ff-4e4189e2d62b, kind: Internal } }), Pool { name: ZpoolName { id: b462a7f7-b628-40fe-80ff-4e4189e2d62b, kind: Internal }, info: ZpoolInfo { name: "oxi_b462a7f7-b628-40fe-80ff-4e4189e2d62b", size: 20937965568, allocated: 540672, free: 20937424896, health: Online }, parent: DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-b462a7f7-b628-40fe-80ff-4e4189e2d62b", model: "synthetic-model" } })} }
552023-12-06T17:40:39.331ZINFOSledAgent: Starting Bootstore file = sled-agent/src/long_running_tasks.rs:200
562023-12-06T17:40:39.331ZWARNSledAgent (bootstore): No ledger in /pool/int/a462a7f7-b628-40fe-80ff-4e4189e2d62b/cluster/bootstore-fsm-state.json file = common/src/ledger.rs:187 peer_id = pc-c6eef3d4b5d9454d80e3983e3692cc58-i86pc
572023-12-06T17:40:39.331ZERROSledAgent (bootstore): Failed to read ledger: Not found in storage file = common/src/ledger.rs:91 path = /pool/int/a462a7f7-b628-40fe-80ff-4e4189e2d62b/cluster/bootstore-fsm-state.json peer_id = pc-c6eef3d4b5d9454d80e3983e3692cc58-i86pc
582023-12-06T17:40:39.331ZWARNSledAgent (bootstore): No ledger in /pool/int/b462a7f7-b628-40fe-80ff-4e4189e2d62b/cluster/bootstore-fsm-state.json file = common/src/ledger.rs:187 peer_id = pc-c6eef3d4b5d9454d80e3983e3692cc58-i86pc
592023-12-06T17:40:39.331ZERROSledAgent (bootstore): Failed to read ledger: Not found in storage file = common/src/ledger.rs:91 path = /pool/int/b462a7f7-b628-40fe-80ff-4e4189e2d62b/cluster/bootstore-fsm-state.json peer_id = pc-c6eef3d4b5d9454d80e3983e3692cc58-i86pc
602023-12-06T17:40:39.331ZINFOSledAgent (bootstore): No ledger found. Loading Fsm::State as Uninitialized file = bootstore/src/schemes/v0/storage.rs:83 peer_id = pc-c6eef3d4b5d9454d80e3983e3692cc58-i86pc
612023-12-06T17:40:39.331ZWARNSledAgent (bootstore): No ledger in /pool/int/a462a7f7-b628-40fe-80ff-4e4189e2d62b/cluster/bootstore-network-config.json file = common/src/ledger.rs:187 peer_id = pc-c6eef3d4b5d9454d80e3983e3692cc58-i86pc
622023-12-06T17:40:39.331ZERROSledAgent (bootstore): Failed to read ledger: Not found in storage file = common/src/ledger.rs:91 path = /pool/int/a462a7f7-b628-40fe-80ff-4e4189e2d62b/cluster/bootstore-network-config.json peer_id = pc-c6eef3d4b5d9454d80e3983e3692cc58-i86pc
632023-12-06T17:40:39.331ZWARNSledAgent (bootstore): No ledger in /pool/int/b462a7f7-b628-40fe-80ff-4e4189e2d62b/cluster/bootstore-network-config.json file = common/src/ledger.rs:187 peer_id = pc-c6eef3d4b5d9454d80e3983e3692cc58-i86pc
642023-12-06T17:40:39.331ZERROSledAgent (bootstore): Failed to read ledger: Not found in storage file = common/src/ledger.rs:91 path = /pool/int/b462a7f7-b628-40fe-80ff-4e4189e2d62b/cluster/bootstore-network-config.json peer_id = pc-c6eef3d4b5d9454d80e3983e3692cc58-i86pc
652023-12-06T17:40:39.331ZINFOSledAgent (bootstore): No ledger found for network config file = bootstore/src/schemes/v0/storage.rs:146 peer_id = pc-c6eef3d4b5d9454d80e3983e3692cc58-i86pc
662023-12-06T17:40:39.331ZINFOSledAgent: Starting Bootstore DDMD poller file = sled-agent/src/long_running_tasks.rs:205
672023-12-06T17:40:39.331ZINFOSledAgent: Starting ZoneBundler related tasks file = sled-agent/src/long_running_tasks.rs:220
682023-12-06T17:40:39.345ZINFOSledAgent (ServiceManager): Creating ServiceManager file = sled-agent/src/services.rs:575
692023-12-06T17:40:39.345ZINFOSledAgent (auto-cleanup-task): top of bundle cleanup loop file = sled-agent/src/zone_bundle.rs:288 next_cleanup = Instant { tv_sec: 5747, tv_nsec: 296746330 } time_to_next_cleanup = 599.999937478s
702023-12-06T17:40:39.345ZINFOSledAgent (ServiceManager): Ensuring scrimlet services (enabling services) file = sled-agent/src/services.rs:3042
712023-12-06T17:40:39.345ZINFOSledAgent (StorageManager): Received GetLatestResources(Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) }) file = sled-storage/src/manager.rs:346
722023-12-06T17:40:39.347ZWARNSledAgent (BootstrapAgentStartup): No ledger in /pool/int/a462a7f7-b628-40fe-80ff-4e4189e2d62b/config/sled-agent-request.json file = common/src/ledger.rs:187
732023-12-06T17:40:39.348ZERROSledAgent (BootstrapAgentStartup): Failed to read ledger: Not found in storage file = common/src/ledger.rs:91 path = /pool/int/a462a7f7-b628-40fe-80ff-4e4189e2d62b/config/sled-agent-request.json
742023-12-06T17:40:39.348ZWARNSledAgent (BootstrapAgentStartup): No ledger in /pool/int/b462a7f7-b628-40fe-80ff-4e4189e2d62b/config/sled-agent-request.json file = common/src/ledger.rs:187
752023-12-06T17:40:39.348ZERROSledAgent (BootstrapAgentStartup): Failed to read ledger: Not found in storage file = common/src/ledger.rs:91 path = /pool/int/b462a7f7-b628-40fe-80ff-4e4189e2d62b/config/sled-agent-request.json
762023-12-06T17:40:39.349ZINFOSledAgent (dropshot (BootstrapAgent)): listening file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:195 local_addr = [fdb0:18c0:4d0d:a02a::1]:80
772023-12-06T17:40:39.349ZINFOSledAgent (SledAgentSprocketsServer): Started listening file = sled-agent/src/bootstrap/sprockets_server.rs:44 local_addr = [fdb0:18c0:4d0d:a02a::1]:12346
782023-12-06T17:40:39.349ZINFOSledAgent (RSS): Injecting RSS configuration: RackInitializeRequest { rack_subnet: fd00:1122:3344:100::, trust_quorum_peers: None, bootstrap_discovery: OnlyOurs, ntp_servers: [ "0.pool.ntp.org", ], dns_servers: [ 1.1.1.1, 9.9.9.9, ], internal_services_ip_pool_ranges: [ V4( Ipv4Range { first: 10.151.2.164, last: 10.151.2.173, }, ), ], external_dns_ips: [ 10.151.2.164, 10.151.2.165, ], external_dns_zone_name: "oxide.test", external_certificates: "<redacted>", recovery_silo: RecoverySiloConfig { silo_name: Name( "recovery", ), user_name: UserId( "recovery", ), user_password_hash: NewPasswordHash( PasswordHashString { string: "$argon2id$v=19$m=98304,t=13,p=1$RUlWc0ZxaHo0WFdrN0N6ZQ$S8p52j85GPvMhR/ek3GL0el/oProgTwWpHJZ8lsQQoY", encoding: B64, }, ), }, rack_network_config: Some( RackNetworkConfigV1 { rack_subnet: Ipv6Network { addr: fd00:1122:3344:1::, prefix: 56, }, infra_ip_first: 10.151.2.174, infra_ip_last: 10.151.2.174, ports: [ PortConfigV1 { routes: [ RouteConfig { destination: V4( Ipv4Network { addr: 0.0.0.0, prefix: 0, }, ), nexthop: 10.151.2.100, }, ], addresses: [ V4( Ipv4Network { addr: 10.151.2.174, prefix: 32, }, ), ], switch: Switch0, port: "qsfp0", uplink_port_speed: Speed40G, uplink_port_fec: None, bgp_peers: [], autoneg: false, }, ], bgp: [], }, ), } file = sled-agent/src/rack_setup/service.rs:803
792023-12-06T17:40:39.349ZINFOSledAgent (DnsResolver): new DNS resolver addresses = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53, [fd00:1122:3344:4::1]:53, [fd00:1122:3344:5::1]:53] file = internal-dns/src/resolver.rs:60
802023-12-06T17:40:39.349ZINFOSledAgent (StorageManager): Received GetLatestResources(Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) }) file = sled-storage/src/manager.rs:346
812023-12-06T17:40:39.349ZWARNSledAgent (RSS): No ledger in /pool/int/a462a7f7-b628-40fe-80ff-4e4189e2d62b/config/rss-plan-completed.marker file = common/src/ledger.rs:187
822023-12-06T17:40:39.349ZERROSledAgent (RSS): Failed to read ledger: Not found in storage file = common/src/ledger.rs:91 path = /pool/int/a462a7f7-b628-40fe-80ff-4e4189e2d62b/config/rss-plan-completed.marker
832023-12-06T17:40:39.349ZWARNSledAgent (RSS): No ledger in /pool/int/b462a7f7-b628-40fe-80ff-4e4189e2d62b/config/rss-plan-completed.marker file = common/src/ledger.rs:187
842023-12-06T17:40:39.349ZERROSledAgent (RSS): Failed to read ledger: Not found in storage file = common/src/ledger.rs:91 path = /pool/int/b462a7f7-b628-40fe-80ff-4e4189e2d62b/config/rss-plan-completed.marker
852023-12-06T17:40:39.349ZINFOSledAgent (RSS): RSS configuration has not been fully applied yet file = sled-agent/src/rack_setup/service.rs:863
862023-12-06T17:40:39.349ZINFOSledAgent (StorageManager): Received GetLatestResources(Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) }) file = sled-storage/src/manager.rs:346
872023-12-06T17:40:39.349ZWARNSledAgent (RSS): No ledger in /pool/int/a462a7f7-b628-40fe-80ff-4e4189e2d62b/config/rss-sled-plan.json file = common/src/ledger.rs:187
882023-12-06T17:40:39.349ZERROSledAgent (RSS): Failed to read ledger: Not found in storage file = common/src/ledger.rs:91 path = /pool/int/a462a7f7-b628-40fe-80ff-4e4189e2d62b/config/rss-sled-plan.json
892023-12-06T17:40:39.349ZWARNSledAgent (RSS): No ledger in /pool/int/b462a7f7-b628-40fe-80ff-4e4189e2d62b/config/rss-sled-plan.json file = common/src/ledger.rs:187
902023-12-06T17:40:39.349ZERROSledAgent (RSS): Failed to read ledger: Not found in storage file = common/src/ledger.rs:91 path = /pool/int/b462a7f7-b628-40fe-80ff-4e4189e2d62b/config/rss-sled-plan.json
912023-12-06T17:40:39.349ZINFOSledAgent (RSS): Creating new allocation plan file = sled-agent/src/rack_setup/service.rs:904
922023-12-06T17:40:39.349ZINFOSledAgent (RSS): Serializing plan file = sled-agent/src/rack_setup/plan/sled.rs:118
932023-12-06T17:40:39.349ZINFOSledAgent (RSS): Creating plan for the sled at fdb0:18c0:4d0d:a02a::1 file = sled-agent/src/rack_setup/plan/sled.rs:91
942023-12-06T17:40:39.349ZINFOSledAgent (StorageManager): Received GetLatestResources(Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) }) file = sled-storage/src/manager.rs:346
952023-12-06T17:40:39.349ZINFOSledAgent (RSS): Writing ledger to /pool/int/a462a7f7-b628-40fe-80ff-4e4189e2d62b/config/.rss-sled-plan.json.tmp file = common/src/ledger.rs:198
962023-12-06T17:40:39.350ZINFOSledAgent (RSS): Writing ledger to /pool/int/b462a7f7-b628-40fe-80ff-4e4189e2d62b/config/.rss-sled-plan.json.tmp file = common/src/ledger.rs:198
972023-12-06T17:40:39.350ZINFOSledAgent (RSS): Sled plan written to storage file = sled-agent/src/rack_setup/plan/sled.rs:138
982023-12-06T17:40:39.350ZINFOSledAgent (RSS): Writing Rack Network Configuration to bootstore file = sled-agent/src/rack_setup/service.rs:937
992023-12-06T17:40:39.350ZINFOSledAgent (bootstore): Attempting to update network config with generation: 1, current_generation: 0 file = bootstore/src/schemes/v0/peer.rs:556 peer_id = pc-c6eef3d4b5d9454d80e3983e3692cc58-i86pc
1002023-12-06T17:40:39.350ZINFOSledAgent (bootstore): Writing ledger to /pool/int/a462a7f7-b628-40fe-80ff-4e4189e2d62b/cluster/.bootstore-network-config.json.tmp file = common/src/ledger.rs:198 peer_id = pc-c6eef3d4b5d9454d80e3983e3692cc58-i86pc
1012023-12-06T17:40:39.350ZINFOSledAgent (bootstore): Writing ledger to /pool/int/b462a7f7-b628-40fe-80ff-4e4189e2d62b/cluster/.bootstore-network-config.json.tmp file = common/src/ledger.rs:198 peer_id = pc-c6eef3d4b5d9454d80e3983e3692cc58-i86pc
1022023-12-06T17:40:39.350ZINFOSledAgent (bootstore): Broadcasting network config with generation 1 file = bootstore/src/schemes/v0/peer.rs:640 peer_id = pc-c6eef3d4b5d9454d80e3983e3692cc58-i86pc
1032023-12-06T17:40:39.350ZINFOSledAgent (BootstrapAgentRssHandler): Received initialization request from RSS file = sled-agent/src/bootstrap/rss_handle.rs:217 request = StartSledAgentRequest { generation: 0, schema_version: 1, body: StartSledAgentRequestBody { id: 1e40a2ae-ffda-4a94-95f6-839f3349172b, rack_id: 7487a3c5-0384-47e3-8028-3cee170ed4d7, use_trust_quorum: false, is_lrtq_learner: false, subnet: Ipv6Subnet { net: Ipv6Net(Ipv6Network { addr: fd00:1122:3344:101::, prefix: 64 }) } } } target_sled = [fdb0:18c0:4d0d:a02a::1]:12346
1042023-12-06T17:40:39.351ZINFOSledAgent (SledAgentSprocketsServer): Accepted connection file = sled-agent/src/bootstrap/sprockets_server.rs:66 remote_addr = [fdb0:18c0:4d0d:a02a::1]:45729
1052023-12-06T17:40:39.351ZINFOSledAgent (SledAgentMain): Loading Sled Agent: StartSledAgentRequest { generation: 0, schema_version: 1, body: StartSledAgentRequestBody { id: 1e40a2ae-ffda-4a94-95f6-839f3349172b, rack_id: 7487a3c5-0384-47e3-8028-3cee170ed4d7, use_trust_quorum: false, is_lrtq_learner: false, subnet: Ipv6Subnet { net: Ipv6Net(Ipv6Network { addr: fd00:1122:3344:101::, prefix: 64 }) } } } file = sled-agent/src/bootstrap/server.rs:364
1062023-12-06T17:40:39.351ZINFOSledAgent (SledAgentMain): KeyManager: using hardcoded secret retriever file = sled-agent/src/bootstrap/server.rs:379
1072023-12-06T17:40:39.351ZINFOSledAgent: setting up sled agent server file = sled-agent/src/server.rs:47
1082023-12-06T17:40:39.351ZINFOSledAgent (DnsResolver): new DNS resolver addresses = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53, [fd00:1122:3344:4::1]:53, [fd00:1122:3344:5::1]:53] file = internal-dns/src/resolver.rs:60
1092023-12-06T17:40:39.351ZINFOSledAgent (StorageManager): Received KeyManagerReady file = sled-storage/src/manager.rs:346
1102023-12-06T17:40:39.351ZINFOSledAgent (StorageManager): Attempting to add queued disks file = sled-storage/src/manager.rs:394 num_disks = 9
1112023-12-06T17:40:39.353ZINFOSledAgent (ServiceManager): Enabling switch zone (new) file = sled-agent/src/services.rs:3270
1122023-12-06T17:40:39.360ZINFOSledAgent (StorageManager): Loading latest secret disk_id = DiskIdentity {\n vendor: "synthetic-vendor",\n serial: "synthetic-serial-d462a7f7-b628-40fe-80ff-4e4189e2d62b",\n model: "synthetic-model",\n} file = sled-storage/src/dataset.rs:261
1132023-12-06T17:40:39.360ZINFOSledAgent (StorageManager): Loaded latest secret disk_id = DiskIdentity {\n vendor: "synthetic-vendor",\n serial: "synthetic-serial-d462a7f7-b628-40fe-80ff-4e4189e2d62b",\n model: "synthetic-model",\n} epoch = 0 file = sled-storage/src/dataset.rs:263
1142023-12-06T17:40:39.360ZINFOSledAgent (StorageManager): Retrieving key disk_id = DiskIdentity {\n vendor: "synthetic-vendor",\n serial: "synthetic-serial-d462a7f7-b628-40fe-80ff-4e4189e2d62b",\n model: "synthetic-model",\n} epoch = 0 file = sled-storage/src/dataset.rs:267
1152023-12-06T17:40:39.360ZINFOSledAgent (StorageManager): Got key disk_id = DiskIdentity {\n vendor: "synthetic-vendor",\n serial: "synthetic-serial-d462a7f7-b628-40fe-80ff-4e4189e2d62b",\n model: "synthetic-model",\n} epoch = 0 file = sled-storage/src/dataset.rs:269
1162023-12-06T17:40:39.360ZINFOSledAgent (StorageManager): Created keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-d462a7f7-b628-40fe-80ff-4e4189e2d62b-synthetic-model-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:37
1172023-12-06T17:40:39.360ZINFOSledAgent (StorageManager): Ensuring encrypted filesystem: crypt for epoch 0 file = sled-storage/src/dataset.rs:281
1182023-12-06T17:40:39.391ZINFOSledAgent (StorageManager): Zeroed and unlinked keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-d462a7f7-b628-40fe-80ff-4e4189e2d62b-synthetic-model-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:56
1192023-12-06T17:40:39.399ZINFOSledAgent (StorageManager): Automatically destroying dataset: oxp_d462a7f7-b628-40fe-80ff-4e4189e2d62b/crypt/zone file = sled-storage/src/dataset.rs:323
1202023-12-06T17:40:39.402ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {}
1212023-12-06T17:40:39.420ZINFOSledAgent: SledAgent::new(..) starting file = sled-agent/src/sled_agent.rs:305 sled_id = 1e40a2ae-ffda-4a94-95f6-839f3349172b
1222023-12-06T17:40:39.420ZINFOSledAgent (BootstrapAgentStartup): Sending prefix to ddmd for advertisement DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:91 prefix = Ipv6Prefix { addr: fd00:1122:3344:101::, len: 64 }
1232023-12-06T17:40:39.519ZINFOSledAgent (StorageManager): Loading latest secret disk_id = DiskIdentity {\n vendor: "synthetic-vendor",\n serial: "synthetic-serial-f4b4dc87-ab46-49fb-a4b4-d361ae214c03",\n model: "synthetic-model",\n} file = sled-storage/src/dataset.rs:261
1242023-12-06T17:40:39.519ZINFOSledAgent (StorageManager): Loaded latest secret disk_id = DiskIdentity {\n vendor: "synthetic-vendor",\n serial: "synthetic-serial-f4b4dc87-ab46-49fb-a4b4-d361ae214c03",\n model: "synthetic-model",\n} epoch = 0 file = sled-storage/src/dataset.rs:263
1252023-12-06T17:40:39.519ZINFOSledAgent (StorageManager): Retrieving key disk_id = DiskIdentity {\n vendor: "synthetic-vendor",\n serial: "synthetic-serial-f4b4dc87-ab46-49fb-a4b4-d361ae214c03",\n model: "synthetic-model",\n} epoch = 0 file = sled-storage/src/dataset.rs:267
1262023-12-06T17:40:39.519ZINFOSledAgent (StorageManager): Got key disk_id = DiskIdentity {\n vendor: "synthetic-vendor",\n serial: "synthetic-serial-f4b4dc87-ab46-49fb-a4b4-d361ae214c03",\n model: "synthetic-model",\n} epoch = 0 file = sled-storage/src/dataset.rs:269
1272023-12-06T17:40:39.520ZINFOSledAgent (StorageManager): Created keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-f4b4dc87-ab46-49fb-a4b4-d361ae214c03-synthetic-model-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:37
1282023-12-06T17:40:39.520ZINFOSledAgent (StorageManager): Ensuring encrypted filesystem: crypt for epoch 0 file = sled-storage/src/dataset.rs:281
1292023-12-06T17:40:39.546ZINFOSledAgent (StorageManager): Zeroed and unlinked keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-f4b4dc87-ab46-49fb-a4b4-d361ae214c03-synthetic-model-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:56
1302023-12-06T17:40:39.553ZINFOSledAgent (StorageManager): Automatically destroying dataset: oxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone file = sled-storage/src/dataset.rs:323
1312023-12-06T17:40:39.672ZINFOSledAgent (StorageManager): Loading latest secret disk_id = DiskIdentity {\n vendor: "synthetic-vendor",\n serial: "synthetic-serial-24b4dc87-ab46-49fb-a4b4-d361ae214c03",\n model: "synthetic-model",\n} file = sled-storage/src/dataset.rs:261
1322023-12-06T17:40:39.672ZINFOSledAgent (StorageManager): Loaded latest secret disk_id = DiskIdentity {\n vendor: "synthetic-vendor",\n serial: "synthetic-serial-24b4dc87-ab46-49fb-a4b4-d361ae214c03",\n model: "synthetic-model",\n} epoch = 0 file = sled-storage/src/dataset.rs:263
1332023-12-06T17:40:39.672ZINFOSledAgent (StorageManager): Retrieving key disk_id = DiskIdentity {\n vendor: "synthetic-vendor",\n serial: "synthetic-serial-24b4dc87-ab46-49fb-a4b4-d361ae214c03",\n model: "synthetic-model",\n} epoch = 0 file = sled-storage/src/dataset.rs:267
1342023-12-06T17:40:39.672ZINFOSledAgent (StorageManager): Got key disk_id = DiskIdentity {\n vendor: "synthetic-vendor",\n serial: "synthetic-serial-24b4dc87-ab46-49fb-a4b4-d361ae214c03",\n model: "synthetic-model",\n} epoch = 0 file = sled-storage/src/dataset.rs:269
1352023-12-06T17:40:39.672ZINFOSledAgent (StorageManager): Created keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-24b4dc87-ab46-49fb-a4b4-d361ae214c03-synthetic-model-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:37
1362023-12-06T17:40:39.672ZINFOSledAgent (StorageManager): Ensuring encrypted filesystem: crypt for epoch 0 file = sled-storage/src/dataset.rs:281
1372023-12-06T17:40:39.699ZINFOSledAgent (StorageManager): Zeroed and unlinked keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-24b4dc87-ab46-49fb-a4b4-d361ae214c03-synthetic-model-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:56
1382023-12-06T17:40:39.707ZINFOSledAgent (StorageManager): Automatically destroying dataset: oxp_24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone file = sled-storage/src/dataset.rs:323
1392023-12-06T17:40:39.825ZINFOSledAgent (StorageManager): Loading latest secret disk_id = DiskIdentity {\n vendor: "synthetic-vendor",\n serial: "synthetic-serial-14b4dc87-ab46-49fb-a4b4-d361ae214c03",\n model: "synthetic-model",\n} file = sled-storage/src/dataset.rs:261
1402023-12-06T17:40:39.825ZINFOSledAgent (StorageManager): Loaded latest secret disk_id = DiskIdentity {\n vendor: "synthetic-vendor",\n serial: "synthetic-serial-14b4dc87-ab46-49fb-a4b4-d361ae214c03",\n model: "synthetic-model",\n} epoch = 0 file = sled-storage/src/dataset.rs:263
1412023-12-06T17:40:39.825ZINFOSledAgent (StorageManager): Retrieving key disk_id = DiskIdentity {\n vendor: "synthetic-vendor",\n serial: "synthetic-serial-14b4dc87-ab46-49fb-a4b4-d361ae214c03",\n model: "synthetic-model",\n} epoch = 0 file = sled-storage/src/dataset.rs:267
1422023-12-06T17:40:39.825ZINFOSledAgent (StorageManager): Got key disk_id = DiskIdentity {\n vendor: "synthetic-vendor",\n serial: "synthetic-serial-14b4dc87-ab46-49fb-a4b4-d361ae214c03",\n model: "synthetic-model",\n} epoch = 0 file = sled-storage/src/dataset.rs:269
1432023-12-06T17:40:39.825ZINFOSledAgent (StorageManager): Created keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-14b4dc87-ab46-49fb-a4b4-d361ae214c03-synthetic-model-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:37
1442023-12-06T17:40:39.825ZINFOSledAgent (StorageManager): Ensuring encrypted filesystem: crypt for epoch 0 file = sled-storage/src/dataset.rs:281
1452023-12-06T17:40:39.853ZINFOSledAgent (StorageManager): Zeroed and unlinked keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-14b4dc87-ab46-49fb-a4b4-d361ae214c03-synthetic-model-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:56
1462023-12-06T17:40:39.860ZINFOSledAgent (StorageManager): Automatically destroying dataset: oxp_14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone file = sled-storage/src/dataset.rs:323
1472023-12-06T17:40:39.977ZINFOSledAgent (StorageManager): Loading latest secret disk_id = DiskIdentity {\n vendor: "synthetic-vendor",\n serial: "synthetic-serial-e4b4dc87-ab46-49fb-a4b4-d361ae214c03",\n model: "synthetic-model",\n} file = sled-storage/src/dataset.rs:261
1482023-12-06T17:40:39.977ZINFOSledAgent (StorageManager): Loaded latest secret disk_id = DiskIdentity {\n vendor: "synthetic-vendor",\n serial: "synthetic-serial-e4b4dc87-ab46-49fb-a4b4-d361ae214c03",\n model: "synthetic-model",\n} epoch = 0 file = sled-storage/src/dataset.rs:263
1492023-12-06T17:40:39.977ZINFOSledAgent (StorageManager): Retrieving key disk_id = DiskIdentity {\n vendor: "synthetic-vendor",\n serial: "synthetic-serial-e4b4dc87-ab46-49fb-a4b4-d361ae214c03",\n model: "synthetic-model",\n} epoch = 0 file = sled-storage/src/dataset.rs:267
1502023-12-06T17:40:39.977ZINFOSledAgent (StorageManager): Got key disk_id = DiskIdentity {\n vendor: "synthetic-vendor",\n serial: "synthetic-serial-e4b4dc87-ab46-49fb-a4b4-d361ae214c03",\n model: "synthetic-model",\n} epoch = 0 file = sled-storage/src/dataset.rs:269
1512023-12-06T17:40:39.977ZINFOSledAgent (StorageManager): Created keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-e4b4dc87-ab46-49fb-a4b4-d361ae214c03-synthetic-model-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:37
1522023-12-06T17:40:39.977ZINFOSledAgent (StorageManager): Ensuring encrypted filesystem: crypt for epoch 0 file = sled-storage/src/dataset.rs:281
1532023-12-06T17:40:40.005ZINFOSledAgent (StorageManager): Zeroed and unlinked keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-e4b4dc87-ab46-49fb-a4b4-d361ae214c03-synthetic-model-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:56
1542023-12-06T17:40:40.018ZINFOSledAgent (StorageManager): Automatically destroying dataset: oxp_e4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone file = sled-storage/src/dataset.rs:323
1552023-12-06T17:40:40.141ZINFOSledAgent (StorageManager): Loading latest secret disk_id = DiskIdentity {\n vendor: "synthetic-vendor",\n serial: "synthetic-serial-ceb4461c-cf56-4719-ad3c-14430bfdfb60",\n model: "synthetic-model",\n} file = sled-storage/src/dataset.rs:261
1562023-12-06T17:40:40.142ZINFOSledAgent (StorageManager): Loaded latest secret disk_id = DiskIdentity {\n vendor: "synthetic-vendor",\n serial: "synthetic-serial-ceb4461c-cf56-4719-ad3c-14430bfdfb60",\n model: "synthetic-model",\n} epoch = 0 file = sled-storage/src/dataset.rs:263
1572023-12-06T17:40:40.142ZINFOSledAgent (StorageManager): Retrieving key disk_id = DiskIdentity {\n vendor: "synthetic-vendor",\n serial: "synthetic-serial-ceb4461c-cf56-4719-ad3c-14430bfdfb60",\n model: "synthetic-model",\n} epoch = 0 file = sled-storage/src/dataset.rs:267
1582023-12-06T17:40:40.142ZINFOSledAgent (StorageManager): Got key disk_id = DiskIdentity {\n vendor: "synthetic-vendor",\n serial: "synthetic-serial-ceb4461c-cf56-4719-ad3c-14430bfdfb60",\n model: "synthetic-model",\n} epoch = 0 file = sled-storage/src/dataset.rs:269
1592023-12-06T17:40:40.142ZINFOSledAgent (StorageManager): Created keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-ceb4461c-cf56-4719-ad3c-14430bfdfb60-synthetic-model-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:37
1602023-12-06T17:40:40.142ZINFOSledAgent (StorageManager): Ensuring encrypted filesystem: crypt for epoch 0 file = sled-storage/src/dataset.rs:281
1612023-12-06T17:40:40.169ZINFOSledAgent (StorageManager): Zeroed and unlinked keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-ceb4461c-cf56-4719-ad3c-14430bfdfb60-synthetic-model-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:56
1622023-12-06T17:40:40.177ZINFOSledAgent (StorageManager): Automatically destroying dataset: oxp_ceb4461c-cf56-4719-ad3c-14430bfdfb60/crypt/zone file = sled-storage/src/dataset.rs:323
1632023-12-06T17:40:40.297ZINFOSledAgent (StorageManager): Loading latest secret disk_id = DiskIdentity {\n vendor: "synthetic-vendor",\n serial: "synthetic-serial-31bd71cd-4736-4a12-a387-9b74b050396f",\n model: "synthetic-model",\n} file = sled-storage/src/dataset.rs:261
1642023-12-06T17:40:40.297ZINFOSledAgent (StorageManager): Loaded latest secret disk_id = DiskIdentity {\n vendor: "synthetic-vendor",\n serial: "synthetic-serial-31bd71cd-4736-4a12-a387-9b74b050396f",\n model: "synthetic-model",\n} epoch = 0 file = sled-storage/src/dataset.rs:263
1652023-12-06T17:40:40.297ZINFOSledAgent (StorageManager): Retrieving key disk_id = DiskIdentity {\n vendor: "synthetic-vendor",\n serial: "synthetic-serial-31bd71cd-4736-4a12-a387-9b74b050396f",\n model: "synthetic-model",\n} epoch = 0 file = sled-storage/src/dataset.rs:267
1662023-12-06T17:40:40.297ZINFOSledAgent (StorageManager): Got key disk_id = DiskIdentity {\n vendor: "synthetic-vendor",\n serial: "synthetic-serial-31bd71cd-4736-4a12-a387-9b74b050396f",\n model: "synthetic-model",\n} epoch = 0 file = sled-storage/src/dataset.rs:269
1672023-12-06T17:40:40.297ZINFOSledAgent (StorageManager): Created keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-31bd71cd-4736-4a12-a387-9b74b050396f-synthetic-model-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:37
1682023-12-06T17:40:40.297ZINFOSledAgent (StorageManager): Ensuring encrypted filesystem: crypt for epoch 0 file = sled-storage/src/dataset.rs:281
1692023-12-06T17:40:40.327ZINFOSledAgent (StorageManager): Zeroed and unlinked keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-31bd71cd-4736-4a12-a387-9b74b050396f-synthetic-model-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:56
1702023-12-06T17:40:40.335ZINFOSledAgent (StorageManager): Automatically destroying dataset: oxp_31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone file = sled-storage/src/dataset.rs:323
1712023-12-06T17:40:40.455ZINFOSledAgent (StorageManager): Loading latest secret disk_id = DiskIdentity {\n vendor: "synthetic-vendor",\n serial: "synthetic-serial-616b26df-e62a-4c68-b506-f4a923d8aaf7",\n model: "synthetic-model",\n} file = sled-storage/src/dataset.rs:261
1722023-12-06T17:40:40.455ZINFOSledAgent (StorageManager): Loaded latest secret disk_id = DiskIdentity {\n vendor: "synthetic-vendor",\n serial: "synthetic-serial-616b26df-e62a-4c68-b506-f4a923d8aaf7",\n model: "synthetic-model",\n} epoch = 0 file = sled-storage/src/dataset.rs:263
1732023-12-06T17:40:40.455ZINFOSledAgent (StorageManager): Retrieving key disk_id = DiskIdentity {\n vendor: "synthetic-vendor",\n serial: "synthetic-serial-616b26df-e62a-4c68-b506-f4a923d8aaf7",\n model: "synthetic-model",\n} epoch = 0 file = sled-storage/src/dataset.rs:267
1742023-12-06T17:40:40.456ZINFOSledAgent (StorageManager): Got key disk_id = DiskIdentity {\n vendor: "synthetic-vendor",\n serial: "synthetic-serial-616b26df-e62a-4c68-b506-f4a923d8aaf7",\n model: "synthetic-model",\n} epoch = 0 file = sled-storage/src/dataset.rs:269
1752023-12-06T17:40:40.456ZINFOSledAgent (StorageManager): Created keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-616b26df-e62a-4c68-b506-f4a923d8aaf7-synthetic-model-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:37
1762023-12-06T17:40:40.456ZINFOSledAgent (StorageManager): Ensuring encrypted filesystem: crypt for epoch 0 file = sled-storage/src/dataset.rs:281
1772023-12-06T17:40:40.483ZINFOSledAgent (StorageManager): Zeroed and unlinked keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-616b26df-e62a-4c68-b506-f4a923d8aaf7-synthetic-model-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:56
1782023-12-06T17:40:40.490ZINFOSledAgent (StorageManager): Automatically destroying dataset: oxp_616b26df-e62a-4c68-b506-f4a923d8aaf7/crypt/zone file = sled-storage/src/dataset.rs:323
1792023-12-06T17:40:40.612ZINFOSledAgent (StorageManager): Loading latest secret disk_id = DiskIdentity {\n vendor: "synthetic-vendor",\n serial: "synthetic-serial-cd70d7f6-2354-4bf2-8012-55bf9eaf7930",\n model: "synthetic-model",\n} file = sled-storage/src/dataset.rs:261
1802023-12-06T17:40:40.613ZINFOSledAgent (StorageManager): Loaded latest secret disk_id = DiskIdentity {\n vendor: "synthetic-vendor",\n serial: "synthetic-serial-cd70d7f6-2354-4bf2-8012-55bf9eaf7930",\n model: "synthetic-model",\n} epoch = 0 file = sled-storage/src/dataset.rs:263
1812023-12-06T17:40:40.613ZINFOSledAgent (StorageManager): Retrieving key disk_id = DiskIdentity {\n vendor: "synthetic-vendor",\n serial: "synthetic-serial-cd70d7f6-2354-4bf2-8012-55bf9eaf7930",\n model: "synthetic-model",\n} epoch = 0 file = sled-storage/src/dataset.rs:267
1822023-12-06T17:40:40.613ZINFOSledAgent (StorageManager): Got key disk_id = DiskIdentity {\n vendor: "synthetic-vendor",\n serial: "synthetic-serial-cd70d7f6-2354-4bf2-8012-55bf9eaf7930",\n model: "synthetic-model",\n} epoch = 0 file = sled-storage/src/dataset.rs:269
1832023-12-06T17:40:40.613ZINFOSledAgent (StorageManager): Created keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-cd70d7f6-2354-4bf2-8012-55bf9eaf7930-synthetic-model-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:37
1842023-12-06T17:40:40.613ZINFOSledAgent (StorageManager): Ensuring encrypted filesystem: crypt for epoch 0 file = sled-storage/src/dataset.rs:281
1852023-12-06T17:40:40.641ZINFOSledAgent (StorageManager): Zeroed and unlinked keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-cd70d7f6-2354-4bf2-8012-55bf9eaf7930-synthetic-model-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:56
1862023-12-06T17:40:40.649ZINFOSledAgent (StorageManager): Automatically destroying dataset: oxp_cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone file = sled-storage/src/dataset.rs:323
1872023-12-06T17:40:40.759ZINFOSledAgent (StorageManager): Received GetLatestResources(Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) }) file = sled-storage/src/manager.rs:346
1882023-12-06T17:40:40.759ZINFOSledAgent (StorageManager): Received GetLatestResources(Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) }) file = sled-storage/src/manager.rs:346
1892023-12-06T17:40:40.759ZINFOSledAgent: Requested swap device of size 64 GiB file = sled-agent/src/sled_agent.rs:317 sled_id = 1e40a2ae-ffda-4a94-95f6-839f3349172b
1902023-12-06T17:40:40.764ZINFOSledAgent (StorageMonitor): Received storage manager update file = sled-agent/src/storage_monitor.rs:111 resources = StorageResources { disks: {DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-14b4dc87-ab46-49fb-a4b4-d361ae214c03", model: "synthetic-model" }: (Synthetic(SyntheticDisk { identity: DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-14b4dc87-ab46-49fb-a4b4-d361ae214c03", model: "synthetic-model" }, zpool_name: ZpoolName { id: 14b4dc87-ab46-49fb-a4b4-d361ae214c03, kind: External } }), Pool { name: ZpoolName { id: 14b4dc87-ab46-49fb-a4b4-d361ae214c03, kind: External }, info: ZpoolInfo { name: "oxp_14b4dc87-ab46-49fb-a4b4-d361ae214c03", size: 20937965568, allocated: 823296, free: 20937142272, health: Online }, parent: DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-14b4dc87-ab46-49fb-a4b4-d361ae214c03", model: "synthetic-model" } }), DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-24b4dc87-ab46-49fb-a4b4-d361ae214c03", model: "synthetic-model" }: (Synthetic(SyntheticDisk { identity: DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-24b4dc87-ab46-49fb-a4b4-d361ae214c03", model: "synthetic-model" }, zpool_name: ZpoolName { id: 24b4dc87-ab46-49fb-a4b4-d361ae214c03, kind: External } }), Pool { name: ZpoolName { id: 24b4dc87-ab46-49fb-a4b4-d361ae214c03, kind: External }, info: ZpoolInfo { name: "oxp_24b4dc87-ab46-49fb-a4b4-d361ae214c03", size: 20937965568, allocated: 780288, free: 20937185280, health: Online }, parent: DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-24b4dc87-ab46-49fb-a4b4-d361ae214c03", model: "synthetic-model" } }), DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-31bd71cd-4736-4a12-a387-9b74b050396f", model: "synthetic-model" }: (Synthetic(SyntheticDisk { identity: DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-31bd71cd-4736-4a12-a387-9b74b050396f", model: "synthetic-model" }, zpool_name: ZpoolName { id: 31bd71cd-4736-4a12-a387-9b74b050396f, kind: External } }), Pool { name: ZpoolName { id: 31bd71cd-4736-4a12-a387-9b74b050396f, kind: External }, info: ZpoolInfo { name: "oxp_31bd71cd-4736-4a12-a387-9b74b050396f", size: 20937965568, allocated: 778752, free: 20937186816, health: Online }, parent: DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-31bd71cd-4736-4a12-a387-9b74b050396f", model: "synthetic-model" } }), DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-616b26df-e62a-4c68-b506-f4a923d8aaf7", model: "synthetic-model" }: (Synthetic(SyntheticDisk { identity: DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-616b26df-e62a-4c68-b506-f4a923d8aaf7", model: "synthetic-model" }, zpool_name: ZpoolName { id: 616b26df-e62a-4c68-b506-f4a923d8aaf7, kind: External } }), Pool { name: ZpoolName { id: 616b26df-e62a-4c68-b506-f4a923d8aaf7, kind: External }, info: ZpoolInfo { name: "oxp_616b26df-e62a-4c68-b506-f4a923d8aaf7", size: 20937965568, allocated: 1651200, free: 20936314368, health: Online }, parent: DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-616b26df-e62a-4c68-b506-f4a923d8aaf7", model: "synthetic-model" } }), DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-a462a7f7-b628-40fe-80ff-4e4189e2d62b", model: "synthetic-model" }: (Synthetic(SyntheticDisk { identity: DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-a462a7f7-b628-40fe-80ff-4e4189e2d62b", model: "synthetic-model" }, zpool_name: ZpoolName { id: a462a7f7-b628-40fe-80ff-4e4189e2d62b, kind: Internal } }), Pool { name: ZpoolName { id: a462a7f7-b628-40fe-80ff-4e4189e2d62b, kind: Internal }, info: ZpoolInfo { name: "oxi_a462a7f7-b628-40fe-80ff-4e4189e2d62b", size: 20937965568, allocated: 494592, free: 20937470976, health: Online }, parent: DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-a462a7f7-b628-40fe-80ff-4e4189e2d62b", model: "synthetic-model" } }), DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-b462a7f7-b628-40fe-80ff-4e4189e2d62b", model: "synthetic-model" }: (Synthetic(SyntheticDisk { identity: DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-b462a7f7-b628-40fe-80ff-4e4189e2d62b", model: "synthetic-model" }, zpool_name: ZpoolName { id: b462a7f7-b628-40fe-80ff-4e4189e2d62b, kind: Internal } }), Pool { name: ZpoolName { id: b462a7f7-b628-40fe-80ff-4e4189e2d62b, kind: Internal }, info: ZpoolInfo { name: "oxi_b462a7f7-b628-40fe-80ff-4e4189e2d62b", size: 20937965568, allocated: 540672, free: 20937424896, health: Online }, parent: DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-b462a7f7-b628-40fe-80ff-4e4189e2d62b", model: "synthetic-model" } }), DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-cd70d7f6-2354-4bf2-8012-55bf9eaf7930", model: "synthetic-model" }: (Synthetic(SyntheticDisk { identity: DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-cd70d7f6-2354-4bf2-8012-55bf9eaf7930", model: "synthetic-model" }, zpool_name: ZpoolName { id: cd70d7f6-2354-4bf2-8012-55bf9eaf7930, kind: External } }), Pool { name: ZpoolName { id: cd70d7f6-2354-4bf2-8012-55bf9eaf7930, kind: External }, info: ZpoolInfo { name: "oxp_cd70d7f6-2354-4bf2-8012-55bf9eaf7930", size: 20937965568, allocated: 772608, free: 20937192960, health: Online }, parent: DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-cd70d7f6-2354-4bf2-8012-55bf9eaf7930", model: "synthetic-model" } }), DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-ceb4461c-cf56-4719-ad3c-14430bfdfb60", model: "synthetic-model" }: (Synthetic(SyntheticDisk { identity: DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-ceb4461c-cf56-4719-ad3c-14430bfdfb60", model: "synthetic-model" }, zpool_name: ZpoolName { id: ceb4461c-cf56-4719-ad3c-14430bfdfb60, kind: External } }), Pool { name: ZpoolName { id: ceb4461c-cf56-4719-ad3c-14430bfdfb60, kind: External }, info: ZpoolInfo { name: "oxp_ceb4461c-cf56-4719-ad3c-14430bfdfb60", size: 20937965568, allocated: 775680, free: 20937189888, health: Online }, parent: DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-ceb4461c-cf56-4719-ad3c-14430bfdfb60", model: "synthetic-model" } }), DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-d462a7f7-b628-40fe-80ff-4e4189e2d62b", model: "synthetic-model" }: (Synthetic(SyntheticDisk { identity: DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-d462a7f7-b628-40fe-80ff-4e4189e2d62b", model: "synthetic-model" }, zpool_name: ZpoolName { id: d462a7f7-b628-40fe-80ff-4e4189e2d62b, kind: External } }), Pool { name: ZpoolName { id: d462a7f7-b628-40fe-80ff-4e4189e2d62b, kind: External }, info: ZpoolInfo { name: "oxp_d462a7f7-b628-40fe-80ff-4e4189e2d62b", size: 20937965568, allocated: 775680, free: 20937189888, health: Online }, parent: DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-d462a7f7-b628-40fe-80ff-4e4189e2d62b", model: "synthetic-model" } }), DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-e4b4dc87-ab46-49fb-a4b4-d361ae214c03", model: "synthetic-model" }: (Synthetic(SyntheticDisk { identity: DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-e4b4dc87-ab46-49fb-a4b4-d361ae214c03", model: "synthetic-model" }, zpool_name: ZpoolName { id: e4b4dc87-ab46-49fb-a4b4-d361ae214c03, kind: External } }), Pool { name: ZpoolName { id: e4b4dc87-ab46-49fb-a4b4-d361ae214c03, kind: External }, info: ZpoolInfo { name: "oxp_e4b4dc87-ab46-49fb-a4b4-d361ae214c03", size: 20937965568, allocated: 777216, free: 20937188352, health: Online }, parent: DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-e4b4dc87-ab46-49fb-a4b4-d361ae214c03", model: "synthetic-model" } }), DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-f4b4dc87-ab46-49fb-a4b4-d361ae214c03", model: "synthetic-model" }: (Synthetic(SyntheticDisk { identity: DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-f4b4dc87-ab46-49fb-a4b4-d361ae214c03", model: "synthetic-model" }, zpool_name: ZpoolName { id: f4b4dc87-ab46-49fb-a4b4-d361ae214c03, kind: External } }), Pool { name: ZpoolName { id: f4b4dc87-ab46-49fb-a4b4-d361ae214c03, kind: External }, info: ZpoolInfo { name: "oxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03", size: 20937965568, allocated: 1211904, free: 20936753664, health: Online }, parent: DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-f4b4dc87-ab46-49fb-a4b4-d361ae214c03", model: "synthetic-model" } })} }
1912023-12-06T17:40:40.781ZINFOSledAgent: attempting to create encrypted zvol: name="oxi_a462a7f7-b628-40fe-80ff-4e4189e2d62b/swap", size_gb=64 file = sled-agent/src/swap_device.rs:153
1922023-12-06T17:40:40.784ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_switch file = illumos-utils/src/zone.rs:314
1932023-12-06T17:40:40.802ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_switch file = illumos-utils/src/zone.rs:353
1942023-12-06T17:40:40.804ZINFOSledAgent: successfully created encrypted zvol: name="oxi_a462a7f7-b628-40fe-80ff-4e4189e2d62b/swap", size_gb=64 file = sled-agent/src/swap_device.rs:249
1952023-12-06T17:40:40.804ZINFOSledAgent: adding swap device: swapname="/dev/zvol/dsk/oxi_a462a7f7-b628-40fe-80ff-4e4189e2d62b/swap" file = sled-agent/src/swap_device.rs:96
1962023-12-06T17:40:40.809ZINFOSledAgent: Mounting backing filesystems file = sled-agent/src/sled_agent.rs:332 sled_id = 1e40a2ae-ffda-4a94-95f6-839f3349172b
1972023-12-06T17:40:40.809ZINFOSledAgent (BackingFs): Processing fmd file = sled-agent/src/backing_fs.rs:126
1982023-12-06T17:40:40.809ZINFOSledAgent (BackingFs): Ensuring dataset oxi_a462a7f7-b628-40fe-80ff-4e4189e2d62b/backing/fmd file = sled-agent/src/backing_fs.rs:136
1992023-12-06T17:40:40.872ZINFOSledAgent (BackingFs): Stopping service svc:/system/fmd:default file = sled-agent/src/backing_fs.rs:170
2002023-12-06T17:40:41.090ZINFOSledAgent (BackingFs): Mounting oxi_a462a7f7-b628-40fe-80ff-4e4189e2d62b/backing/fmd on /var/fm/fmd file = sled-agent/src/backing_fs.rs:178
2012023-12-06T17:40:41.102ZINFOSledAgent (BackingFs): Ensuring directory /var/fm/fmd/rsrc file = sled-agent/src/backing_fs.rs:186
2022023-12-06T17:40:41.102ZINFOSledAgent (BackingFs): Ensuring directory /var/fm/fmd/ckpt file = sled-agent/src/backing_fs.rs:186
2032023-12-06T17:40:41.102ZINFOSledAgent (BackingFs): Ensuring directory /var/fm/fmd/xprt file = sled-agent/src/backing_fs.rs:186
2042023-12-06T17:40:41.102ZINFOSledAgent (BackingFs): Starting service svc:/system/fmd:default file = sled-agent/src/backing_fs.rs:192
2052023-12-06T17:40:41.266ZINFOSledAgent: Ensuring contract reaper thread file = illumos-utils/src/running_zone.rs:162
2062023-12-06T17:40:41.791ZINFOSledAgent: using '[AddrObject { interface: "net0", name: "ll" }, AddrObject { interface: "net1", name: "ll" }]' as data links for xde driver file = illumos-utils/src/opte/illumos.rs:80 sled_id = 1e40a2ae-ffda-4a94-95f6-839f3349172b
2072023-12-06T17:40:41.792ZINFOSledAgent (StorageMonitor): Underlay Available file = sled-agent/src/storage_monitor.rs:122 sled_id = 1e40a2ae-ffda-4a94-95f6-839f3349172b
2082023-12-06T17:40:41.792ZINFOSledAgent (InstanceManager): 50% of 102952026112 physical ram = 51476013056 bytes) file = sled-agent/src/instance_manager.rs:177
2092023-12-06T17:40:41.792ZINFOSledAgent (InstanceManager): Setting reservoir size to 49090 MiB bytes file = sled-agent/src/instance_manager.rs:185
2102023-12-06T17:40:41.801ZINFOSledAgent (StorageMonitor): failed to notify nexus about Put(PhysicalDiskPutRequest { model: "synthetic-model", serial: "synthetic-serial-14b4dc87-ab46-49fb-a4b4-d361ae214c03", sled_id: 1e40a2ae-ffda-4a94-95f6-839f3349172b, variant: U2, vendor: "synthetic-vendor" }) err = "Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/physical-disk): error trying to connect: dns error: proto error: io error: No route to host (os error 148)" file = sled-agent/src/storage_monitor.rs:219
2112023-12-06T17:40:41.804ZINFOSledAgent (StorageMonitor): failed to notify nexus about Put(PhysicalDiskPutRequest { model: "synthetic-model", serial: "synthetic-serial-cd70d7f6-2354-4bf2-8012-55bf9eaf7930", sled_id: 1e40a2ae-ffda-4a94-95f6-839f3349172b, variant: U2, vendor: "synthetic-vendor" }) err = "Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/physical-disk): error trying to connect: dns error: proto error: io error: No route to host (os error 148)" file = sled-agent/src/storage_monitor.rs:219
2122023-12-06T17:40:41.804ZINFOSledAgent (StorageMonitor): failed to notify nexus about Put(PhysicalDiskPutRequest { model: "synthetic-model", serial: "synthetic-serial-e4b4dc87-ab46-49fb-a4b4-d361ae214c03", sled_id: 1e40a2ae-ffda-4a94-95f6-839f3349172b, variant: U2, vendor: "synthetic-vendor" }) err = "Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/physical-disk): error trying to connect: dns error: proto error: io error: No route to host (os error 148)" file = sled-agent/src/storage_monitor.rs:219
2132023-12-06T17:40:41.804ZINFOSledAgent (StorageMonitor): failed to notify nexus about a new pool oxp_24b4dc87-ab46-49fb-a4b4-d361ae214c03 on disk DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-24b4dc87-ab46-49fb-a4b4-d361ae214c03", model: "synthetic-model" } err = "Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/sled-agents/1e40a2ae-ffda-4a94-95f6-839f3349172b/zpools/24b4dc87-ab46-49fb-a4b4-d361ae214c03): error trying to connect: dns error: proto error: io error: No route to host (os error 148)" file = sled-agent/src/storage_monitor.rs:270
2142023-12-06T17:40:41.804ZINFOSledAgent (StorageMonitor): failed to notify nexus about a new pool oxp_616b26df-e62a-4c68-b506-f4a923d8aaf7 on disk DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-616b26df-e62a-4c68-b506-f4a923d8aaf7", model: "synthetic-model" } err = "Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/sled-agents/1e40a2ae-ffda-4a94-95f6-839f3349172b/zpools/616b26df-e62a-4c68-b506-f4a923d8aaf7): error trying to connect: dns error: proto error: io error: No route to host (os error 148)" file = sled-agent/src/storage_monitor.rs:270
2152023-12-06T17:40:41.805ZINFOSledAgent (StorageMonitor): failed to notify nexus about a new pool oxp_ceb4461c-cf56-4719-ad3c-14430bfdfb60 on disk DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-ceb4461c-cf56-4719-ad3c-14430bfdfb60", model: "synthetic-model" } err = "Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/sled-agents/1e40a2ae-ffda-4a94-95f6-839f3349172b/zpools/ceb4461c-cf56-4719-ad3c-14430bfdfb60): error trying to connect: dns error: proto error: io error: No route to host (os error 148)" file = sled-agent/src/storage_monitor.rs:270
2162023-12-06T17:40:41.805ZINFOSledAgent (StorageMonitor): failed to notify nexus about Put(PhysicalDiskPutRequest { model: "synthetic-model", serial: "synthetic-serial-24b4dc87-ab46-49fb-a4b4-d361ae214c03", sled_id: 1e40a2ae-ffda-4a94-95f6-839f3349172b, variant: U2, vendor: "synthetic-vendor" }) err = "Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/physical-disk): error trying to connect: dns error: proto error: io error: No route to host (os error 148)" file = sled-agent/src/storage_monitor.rs:219
2172023-12-06T17:40:41.805ZINFOSledAgent (StorageMonitor): failed to notify nexus about Put(PhysicalDiskPutRequest { model: "synthetic-model", serial: "synthetic-serial-31bd71cd-4736-4a12-a387-9b74b050396f", sled_id: 1e40a2ae-ffda-4a94-95f6-839f3349172b, variant: U2, vendor: "synthetic-vendor" }) err = "Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/physical-disk): error trying to connect: dns error: proto error: io error: No route to host (os error 148)" file = sled-agent/src/storage_monitor.rs:219
2182023-12-06T17:40:41.805ZINFOSledAgent (StorageMonitor): failed to notify nexus about Put(PhysicalDiskPutRequest { model: "synthetic-model", serial: "synthetic-serial-616b26df-e62a-4c68-b506-f4a923d8aaf7", sled_id: 1e40a2ae-ffda-4a94-95f6-839f3349172b, variant: U2, vendor: "synthetic-vendor" }) err = "Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/physical-disk): error trying to connect: dns error: proto error: io error: No route to host (os error 148)" file = sled-agent/src/storage_monitor.rs:219
2192023-12-06T17:40:41.805ZINFOSledAgent (StorageMonitor): failed to notify nexus about Put(PhysicalDiskPutRequest { model: "synthetic-model", serial: "synthetic-serial-a462a7f7-b628-40fe-80ff-4e4189e2d62b", sled_id: 1e40a2ae-ffda-4a94-95f6-839f3349172b, variant: M2, vendor: "synthetic-vendor" }) err = "Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/physical-disk): error trying to connect: dns error: proto error: io error: No route to host (os error 148)" file = sled-agent/src/storage_monitor.rs:219
2202023-12-06T17:40:41.805ZINFOSledAgent (StorageMonitor): failed to notify nexus about Put(PhysicalDiskPutRequest { model: "synthetic-model", serial: "synthetic-serial-b462a7f7-b628-40fe-80ff-4e4189e2d62b", sled_id: 1e40a2ae-ffda-4a94-95f6-839f3349172b, variant: M2, vendor: "synthetic-vendor" }) err = "Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/physical-disk): error trying to connect: dns error: proto error: io error: No route to host (os error 148)" file = sled-agent/src/storage_monitor.rs:219
2212023-12-06T17:40:41.805ZINFOSledAgent (StorageMonitor): failed to notify nexus about Put(PhysicalDiskPutRequest { model: "synthetic-model", serial: "synthetic-serial-ceb4461c-cf56-4719-ad3c-14430bfdfb60", sled_id: 1e40a2ae-ffda-4a94-95f6-839f3349172b, variant: U2, vendor: "synthetic-vendor" }) err = "Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/physical-disk): error trying to connect: dns error: proto error: io error: No route to host (os error 148)" file = sled-agent/src/storage_monitor.rs:219
2222023-12-06T17:40:41.806ZINFOSledAgent (StorageMonitor): failed to notify nexus about Put(PhysicalDiskPutRequest { model: "synthetic-model", serial: "synthetic-serial-d462a7f7-b628-40fe-80ff-4e4189e2d62b", sled_id: 1e40a2ae-ffda-4a94-95f6-839f3349172b, variant: U2, vendor: "synthetic-vendor" }) err = "Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/physical-disk): error trying to connect: dns error: proto error: io error: No route to host (os error 148)" file = sled-agent/src/storage_monitor.rs:219
2232023-12-06T17:40:41.806ZINFOSledAgent (StorageMonitor): failed to notify nexus about Put(PhysicalDiskPutRequest { model: "synthetic-model", serial: "synthetic-serial-f4b4dc87-ab46-49fb-a4b4-d361ae214c03", sled_id: 1e40a2ae-ffda-4a94-95f6-839f3349172b, variant: U2, vendor: "synthetic-vendor" }) err = "Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/physical-disk): error trying to connect: dns error: proto error: io error: No route to host (os error 148)" file = sled-agent/src/storage_monitor.rs:219
2242023-12-06T17:40:41.806ZINFOSledAgent (StorageMonitor): failed to notify nexus about a new pool oxp_14b4dc87-ab46-49fb-a4b4-d361ae214c03 on disk DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-14b4dc87-ab46-49fb-a4b4-d361ae214c03", model: "synthetic-model" } err = "Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/sled-agents/1e40a2ae-ffda-4a94-95f6-839f3349172b/zpools/14b4dc87-ab46-49fb-a4b4-d361ae214c03): error trying to connect: dns error: proto error: io error: No route to host (os error 148)" file = sled-agent/src/storage_monitor.rs:270
2252023-12-06T17:40:41.806ZINFOSledAgent (StorageMonitor): failed to notify nexus about a new pool oxp_31bd71cd-4736-4a12-a387-9b74b050396f on disk DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-31bd71cd-4736-4a12-a387-9b74b050396f", model: "synthetic-model" } err = "Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/sled-agents/1e40a2ae-ffda-4a94-95f6-839f3349172b/zpools/31bd71cd-4736-4a12-a387-9b74b050396f): error trying to connect: dns error: proto error: io error: No route to host (os error 148)" file = sled-agent/src/storage_monitor.rs:270
2262023-12-06T17:40:41.806ZINFOSledAgent (StorageMonitor): failed to notify nexus about a new pool oxi_a462a7f7-b628-40fe-80ff-4e4189e2d62b on disk DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-a462a7f7-b628-40fe-80ff-4e4189e2d62b", model: "synthetic-model" } err = "Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/sled-agents/1e40a2ae-ffda-4a94-95f6-839f3349172b/zpools/a462a7f7-b628-40fe-80ff-4e4189e2d62b): error trying to connect: dns error: proto error: io error: No route to host (os error 148)" file = sled-agent/src/storage_monitor.rs:270
2272023-12-06T17:40:41.806ZINFOSledAgent (StorageMonitor): failed to notify nexus about a new pool oxi_b462a7f7-b628-40fe-80ff-4e4189e2d62b on disk DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-b462a7f7-b628-40fe-80ff-4e4189e2d62b", model: "synthetic-model" } err = "Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/sled-agents/1e40a2ae-ffda-4a94-95f6-839f3349172b/zpools/b462a7f7-b628-40fe-80ff-4e4189e2d62b): error trying to connect: dns error: proto error: io error: No route to host (os error 148)" file = sled-agent/src/storage_monitor.rs:270
2282023-12-06T17:40:41.806ZINFOSledAgent (StorageMonitor): failed to notify nexus about a new pool oxp_cd70d7f6-2354-4bf2-8012-55bf9eaf7930 on disk DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-cd70d7f6-2354-4bf2-8012-55bf9eaf7930", model: "synthetic-model" } err = "Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/sled-agents/1e40a2ae-ffda-4a94-95f6-839f3349172b/zpools/cd70d7f6-2354-4bf2-8012-55bf9eaf7930): error trying to connect: dns error: proto error: io error: No route to host (os error 148)" file = sled-agent/src/storage_monitor.rs:270
2292023-12-06T17:40:41.806ZINFOSledAgent (StorageMonitor): failed to notify nexus about a new pool oxp_d462a7f7-b628-40fe-80ff-4e4189e2d62b on disk DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-d462a7f7-b628-40fe-80ff-4e4189e2d62b", model: "synthetic-model" } err = "Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/sled-agents/1e40a2ae-ffda-4a94-95f6-839f3349172b/zpools/d462a7f7-b628-40fe-80ff-4e4189e2d62b): error trying to connect: dns error: proto error: io error: No route to host (os error 148)" file = sled-agent/src/storage_monitor.rs:270
2302023-12-06T17:40:41.806ZINFOSledAgent (StorageMonitor): failed to notify nexus about a new pool oxp_e4b4dc87-ab46-49fb-a4b4-d361ae214c03 on disk DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-e4b4dc87-ab46-49fb-a4b4-d361ae214c03", model: "synthetic-model" } err = "Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/sled-agents/1e40a2ae-ffda-4a94-95f6-839f3349172b/zpools/e4b4dc87-ab46-49fb-a4b4-d361ae214c03): error trying to connect: dns error: proto error: io error: No route to host (os error 148)" file = sled-agent/src/storage_monitor.rs:270
2312023-12-06T17:40:41.807ZINFOSledAgent (StorageMonitor): failed to notify nexus about a new pool oxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03 on disk DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-f4b4dc87-ab46-49fb-a4b4-d361ae214c03", model: "synthetic-model" } err = "Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/sled-agents/1e40a2ae-ffda-4a94-95f6-839f3349172b/zpools/f4b4dc87-ab46-49fb-a4b4-d361ae214c03): error trying to connect: dns error: proto error: io error: No route to host (os error 148)" file = sled-agent/src/storage_monitor.rs:270
2322023-12-06T17:40:44.103ZINFOSledAgent (ServiceManager): Zone booting file = illumos-utils/src/running_zone.rs:512 zone = oxz_switch
2332023-12-06T17:40:44.427ZWARNSledAgent (ServiceManager): wait for service Some("oxz_switch") failed: Property not found. retry in 66.771904ms file = illumos-utils/src/svc.rs:36 zone = oxz_switch
2342023-12-06T17:40:44.428ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {}
2352023-12-06T17:40:44.637ZWARNSledAgent (ServiceManager): wait for service Some("oxz_switch") failed: Property not found. retry in 132.043538ms file = illumos-utils/src/svc.rs:36 zone = oxz_switch
2362023-12-06T17:40:44.815ZWARNSledAgent (ServiceManager): wait for service Some("oxz_switch") failed: Property not found. retry in 244.944816ms file = illumos-utils/src/svc.rs:36 zone = oxz_switch
2372023-12-06T17:40:45.177ZWARNSledAgent (ServiceManager): wait for service Some("oxz_switch") failed: Property not found. retry in 214.894222ms file = illumos-utils/src/svc.rs:36 zone = oxz_switch
2382023-12-06T17:40:45.413ZWARNSledAgent (ServiceManager): wait for service Some("oxz_switch") failed: Property not found. retry in 780.871257ms file = illumos-utils/src/svc.rs:36 zone = oxz_switch
2392023-12-06T17:40:46.360ZWARNSledAgent (ServiceManager): wait for service Some("oxz_switch") failed: Property not found. retry in 562.170006ms file = illumos-utils/src/svc.rs:36 zone = oxz_switch
2402023-12-06T17:40:47.055ZWARNSledAgent (ServiceManager): wait for service Some("oxz_switch") failed: Property not found. retry in 853.813623ms file = illumos-utils/src/svc.rs:36 zone = oxz_switch
2412023-12-06T17:40:47.995ZWARNSledAgent (ServiceManager): wait for service Some("oxz_switch") failed: Property not found. retry in 1.32705836s file = illumos-utils/src/svc.rs:36 zone = oxz_switch
2422023-12-06T17:40:49.641ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {}
2432023-12-06T17:40:50.344ZINFOSledAgent (ServiceManager): Ensuring tfportrear0_0/ll exists in zone file = sled-agent/src/services.rs:1716
2442023-12-06T17:40:52.242ZINFOSledAgent (ServiceManager): sled agent started file = sled-agent/src/services.rs:938 underlay_address = fd00:1122:3344:101::1
2452023-12-06T17:40:52.242ZINFOSledAgent (DnsResolver): new DNS resolver addresses = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53, [fd00:1122:3344:4::1]:53, [fd00:1122:3344:5::1]:53] file = internal-dns/src/resolver.rs:60
2462023-12-06T17:40:52.297ZINFOSledAgent (dropshot (SledAgent)): listening file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:195 local_addr = [fd00:1122:3344:101::1]:12345
2472023-12-06T17:40:52.297ZINFOSledAgent (SledAgentMain): Sled Agent loaded; recording configuration file = sled-agent/src/bootstrap/server.rs:424
2482023-12-06T17:40:52.297ZINFOSledAgent: contacting server nexus, registering sled baseboard = Baseboard { part_number: "i86pc", revision: 0, serial_number: "c6eef3d4b5d9454d80e3983e3692cc58" } file = sled-agent/src/sled_agent.rs:629 id = 1e40a2ae-ffda-4a94-95f6-839f3349172b sled_id = 1e40a2ae-ffda-4a94-95f6-839f3349172b
2492023-12-06T17:40:52.297ZINFOSledAgent (StorageManager): Received GetLatestResources(Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) }) file = sled-storage/src/manager.rs:346
2502023-12-06T17:40:52.297ZINFOSledAgent (SledAgentMain): Writing ledger to /pool/int/a462a7f7-b628-40fe-80ff-4e4189e2d62b/config/.sled-agent-request.json.tmp file = common/src/ledger.rs:198
2512023-12-06T17:40:52.298ZINFOSledAgent (SledAgentMain): Writing ledger to /pool/int/b462a7f7-b628-40fe-80ff-4e4189e2d62b/config/.sled-agent-request.json.tmp file = common/src/ledger.rs:198
2522023-12-06T17:40:52.298ZINFOSledAgent (HardwareMonitor): Sled Agent Started file = sled-agent/src/hardware_monitor.rs:137
2532023-12-06T17:40:52.298ZINFOSledAgent (HardwareMonitor): Checking current full hardware snapshot file = sled-agent/src/hardware_monitor.rs:241 underlay_network_info = Some((fd00:1122:3344:101::2, Some(RackNetworkConfigV1 { rack_subnet: Ipv6Network { addr: fd00:1122:3344:1::, prefix: 56 }, infra_ip_first: 10.151.2.174, infra_ip_last: 10.151.2.174, ports: [PortConfigV1 { routes: [RouteConfig { destination: V4(Ipv4Network { addr: 0.0.0.0, prefix: 0 }), nexthop: 10.151.2.100 }], addresses: [V4(Ipv4Network { addr: 10.151.2.174, prefix: 32 })], switch: Switch0, port: "qsfp0", uplink_port_speed: Speed40G, uplink_port_fec: None, bgp_peers: [], autoneg: false }], bgp: [] })))
2542023-12-06T17:40:52.298ZINFOSledAgent (ServiceManager): Ensuring scrimlet services (enabling services) file = sled-agent/src/services.rs:3042
2552023-12-06T17:40:52.298ZINFOSledAgent (SledAgentSprocketsServer): Connection closed file = sled-agent/src/bootstrap/sprockets_server.rs:73 remote_addr = [fdb0:18c0:4d0d:a02a::1]:45729
2562023-12-06T17:40:52.302ZINFOSledAgent (BootstrapAgentRssHandler): Peer agent initialized file = sled-agent/src/bootstrap/rss_handle.rs:108 peer = [fdb0:18c0:4d0d:a02a::1]:12346
2572023-12-06T17:40:52.302ZINFOSledAgent (BootstrapAgentRssHandler): Initialized sled agent file = sled-agent/src/bootstrap/rss_handle.rs:232 target_sled = [fdb0:18c0:4d0d:a02a::1]:12346
2582023-12-06T17:40:52.302ZINFOSledAgent (StorageManager): Received GetLatestResources(Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) }) file = sled-storage/src/manager.rs:346
2592023-12-06T17:40:52.302ZWARNSledAgent (RSS): No ledger in /pool/int/a462a7f7-b628-40fe-80ff-4e4189e2d62b/config/rss-service-plan-v2.json file = common/src/ledger.rs:187
2602023-12-06T17:40:52.302ZERROSledAgent (RSS): Failed to read ledger: Not found in storage file = common/src/ledger.rs:91 path = /pool/int/a462a7f7-b628-40fe-80ff-4e4189e2d62b/config/rss-service-plan-v2.json
2612023-12-06T17:40:52.302ZWARNSledAgent (RSS): No ledger in /pool/int/b462a7f7-b628-40fe-80ff-4e4189e2d62b/config/rss-service-plan-v2.json file = common/src/ledger.rs:187
2622023-12-06T17:40:52.302ZERROSledAgent (RSS): Failed to read ledger: Not found in storage file = common/src/ledger.rs:91 path = /pool/int/b462a7f7-b628-40fe-80ff-4e4189e2d62b/config/rss-service-plan-v2.json
2632023-12-06T17:40:52.302ZINFOSledAgent (StorageManager): Received GetLatestResources(Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) }) file = sled-storage/src/manager.rs:346
2642023-12-06T17:40:52.302ZWARNSledAgent: failed to register as a metric producer with Nexus error = "Metric registration error: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/metrics/producers): error trying to connect: dns error: proto error: io error: No route to host (os error 148)" file = sled-agent/src/sled_agent.rs:1060 retry_after = 354.752758ms sled_id = 1e40a2ae-ffda-4a94-95f6-839f3349172b
2652023-12-06T17:40:52.303ZINFOSledAgent: failed to notify nexus about sled agent error = Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/sled-agents/1e40a2ae-ffda-4a94-95f6-839f3349172b): error trying to connect: dns error: proto error: io error: No route to host (os error 148) file = sled-agent/src/sled_agent.rs:666 sled_id = 1e40a2ae-ffda-4a94-95f6-839f3349172b
2662023-12-06T17:40:52.334ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:769 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:56175
2672023-12-06T17:40:52.335ZINFOSledAgent (StorageManager): Received GetLatestResources(Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) }) file = sled-storage/src/manager.rs:346
2682023-12-06T17:40:52.335ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 145 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:56175 req_id = 13f034d3-0780-4590-991c-e1ff8a48a80c response_code = 200 uri = /zpools
2692023-12-06T17:40:52.364ZINFOSledAgent: contacting server nexus, registering sled baseboard = Baseboard { part_number: "i86pc", revision: 0, serial_number: "c6eef3d4b5d9454d80e3983e3692cc58" } file = sled-agent/src/sled_agent.rs:629 id = 1e40a2ae-ffda-4a94-95f6-839f3349172b sled_id = 1e40a2ae-ffda-4a94-95f6-839f3349172b
2702023-12-06T17:40:52.364ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:769 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:59244
2712023-12-06T17:40:52.364ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 78 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:59244 req_id = 2d5c18e1-e460-4757-b369-7bc31cd5874c response_code = 200 uri = /sled-role
2722023-12-06T17:40:52.368ZINFOSledAgent (StorageManager): Received GetLatestResources(Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) }) file = sled-storage/src/manager.rs:346
2732023-12-06T17:40:52.368ZINFOSledAgent (RSS): Writing ledger to /pool/int/a462a7f7-b628-40fe-80ff-4e4189e2d62b/config/.rss-service-plan-v2.json.tmp file = common/src/ledger.rs:198
2742023-12-06T17:40:52.368ZINFOSledAgent (RSS): Writing ledger to /pool/int/b462a7f7-b628-40fe-80ff-4e4189e2d62b/config/.rss-service-plan-v2.json.tmp file = common/src/ledger.rs:198
2752023-12-06T17:40:52.368ZINFOSledAgent (RSS): Service plan written to storage file = sled-agent/src/rack_setup/plan/service.rs:749
2762023-12-06T17:40:52.400ZINFOSledAgent (RSS): attempting to set up sled's Omicron zones: OmicronZonesConfig { generation: Generation(2), zones: [OmicronZoneConfig { id: 00f82355-ab6a-44f8-adeb-55a0298e7676, underlay_address: fd00:1122:3344:1::1, zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 14b4dc87-ab46-49fb-a4b4-d361ae214c03, kind: External } }, http_address: [fd00:1122:3344:1::1]:5353, dns_address: [fd00:1122:3344:1::1]:53, gz_address: fd00:1122:3344:1::2, gz_address_index: 0 } }, OmicronZoneConfig { id: be8d8c76-3f0a-4bd9-8466-47f0b72402ea, underlay_address: fd00:1122:3344:2::1, zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 24b4dc87-ab46-49fb-a4b4-d361ae214c03, kind: External } }, http_address: [fd00:1122:3344:2::1]:5353, dns_address: [fd00:1122:3344:2::1]:53, gz_address: fd00:1122:3344:2::2, gz_address_index: 1 } }, OmicronZoneConfig { id: ac3d1180-e1c3-477d-a802-3f8bd92e2528, underlay_address: fd00:1122:3344:3::1, zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 31bd71cd-4736-4a12-a387-9b74b050396f, kind: External } }, http_address: [fd00:1122:3344:3::1]:5353, dns_address: [fd00:1122:3344:3::1]:53, gz_address: fd00:1122:3344:3::2, gz_address_index: 2 } }] } file = sled-agent/src/rack_setup/service.rs:299 sled_address = [fd00:1122:3344:101::1]:12345
2772023-12-06T17:40:52.401ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:769 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:54836
2782023-12-06T17:40:52.401ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 00f82355-ab6a-44f8-adeb-55a0298e7676, dataset_name: DatasetName { pool_name: ZpoolName { id: 14b4dc87-ab46-49fb-a4b4-d361ae214c03, kind: External }, kind: InternalDns }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } }) file = sled-storage/src/manager.rs:346
2792023-12-06T17:40:52.401ZINFOSledAgent (StorageManager): add_dataset: NewFilesystemRequest { dataset_id: 00f82355-ab6a-44f8-adeb-55a0298e7676, dataset_name: DatasetName { pool_name: ZpoolName { id: 14b4dc87-ab46-49fb-a4b4-d361ae214c03, kind: External }, kind: InternalDns }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } } file = sled-storage/src/manager.rs:564
2802023-12-06T17:40:52.450ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: be8d8c76-3f0a-4bd9-8466-47f0b72402ea, dataset_name: DatasetName { pool_name: ZpoolName { id: 24b4dc87-ab46-49fb-a4b4-d361ae214c03, kind: External }, kind: InternalDns }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } }) file = sled-storage/src/manager.rs:346
2812023-12-06T17:40:52.450ZINFOSledAgent (StorageManager): add_dataset: NewFilesystemRequest { dataset_id: be8d8c76-3f0a-4bd9-8466-47f0b72402ea, dataset_name: DatasetName { pool_name: ZpoolName { id: 24b4dc87-ab46-49fb-a4b4-d361ae214c03, kind: External }, kind: InternalDns }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } } file = sled-storage/src/manager.rs:564
2822023-12-06T17:40:52.498ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: ac3d1180-e1c3-477d-a802-3f8bd92e2528, dataset_name: DatasetName { pool_name: ZpoolName { id: 31bd71cd-4736-4a12-a387-9b74b050396f, kind: External }, kind: InternalDns }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } }) file = sled-storage/src/manager.rs:346
2832023-12-06T17:40:52.498ZINFOSledAgent (StorageManager): add_dataset: NewFilesystemRequest { dataset_id: ac3d1180-e1c3-477d-a802-3f8bd92e2528, dataset_name: DatasetName { pool_name: ZpoolName { id: 31bd71cd-4736-4a12-a387-9b74b050396f, kind: External }, kind: InternalDns }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } } file = sled-storage/src/manager.rs:564
2842023-12-06T17:40:52.522ZINFOSledAgent: contacting server nexus, registering sled baseboard = Baseboard { part_number: "i86pc", revision: 0, serial_number: "c6eef3d4b5d9454d80e3983e3692cc58" } file = sled-agent/src/sled_agent.rs:629 id = 1e40a2ae-ffda-4a94-95f6-839f3349172b sled_id = 1e40a2ae-ffda-4a94-95f6-839f3349172b
2852023-12-06T17:40:52.544ZINFOSledAgent (StorageManager): Received GetLatestResources(Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) }) file = sled-storage/src/manager.rs:346
2862023-12-06T17:40:52.544ZWARNSledAgent (ServiceManager): No ledger in /pool/int/a462a7f7-b628-40fe-80ff-4e4189e2d62b/config/omicron-zones.json file = common/src/ledger.rs:187
2872023-12-06T17:40:52.544ZERROSledAgent (ServiceManager): Failed to read ledger: Not found in storage file = common/src/ledger.rs:91 path = /pool/int/a462a7f7-b628-40fe-80ff-4e4189e2d62b/config/omicron-zones.json
2882023-12-06T17:40:52.544ZWARNSledAgent (ServiceManager): No ledger in /pool/int/b462a7f7-b628-40fe-80ff-4e4189e2d62b/config/omicron-zones.json file = common/src/ledger.rs:187
2892023-12-06T17:40:52.544ZERROSledAgent (ServiceManager): Failed to read ledger: Not found in storage file = common/src/ledger.rs:91 path = /pool/int/b462a7f7-b628-40fe-80ff-4e4189e2d62b/config/omicron-zones.json
2902023-12-06T17:40:52.544ZINFOSledAgent (StorageManager): Received GetLatestResources(Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) }) file = sled-storage/src/manager.rs:346
2912023-12-06T17:40:52.546ZINFOSledAgent (StorageManager): Received GetLatestResources(Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) }) file = sled-storage/src/manager.rs:346
2922023-12-06T17:40:52.548ZINFOSledAgent (StorageManager): Received GetLatestResources(Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) }) file = sled-storage/src/manager.rs:346
2932023-12-06T17:40:52.549ZINFOSledAgent (StorageManager): Received GetLatestResources(Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) }) file = sled-storage/src/manager.rs:346
2942023-12-06T17:40:52.601ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_internal_dns_be8d8c76-3f0a-4bd9-8466-47f0b72402ea file = illumos-utils/src/zone.rs:314
2952023-12-06T17:40:52.604ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_internal_dns_ac3d1180-e1c3-477d-a802-3f8bd92e2528 file = illumos-utils/src/zone.rs:314
2962023-12-06T17:40:52.608ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_internal_dns_00f82355-ab6a-44f8-adeb-55a0298e7676 file = illumos-utils/src/zone.rs:314
2972023-12-06T17:40:52.621ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_internal_dns_be8d8c76-3f0a-4bd9-8466-47f0b72402ea file = illumos-utils/src/zone.rs:353
2982023-12-06T17:40:52.625ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_internal_dns_ac3d1180-e1c3-477d-a802-3f8bd92e2528 file = illumos-utils/src/zone.rs:353
2992023-12-06T17:40:52.629ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_internal_dns_00f82355-ab6a-44f8-adeb-55a0298e7676 file = illumos-utils/src/zone.rs:353
3002023-12-06T17:40:52.659ZWARNSledAgent: failed to register as a metric producer with Nexus error = "Metric registration error: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/metrics/producers): error trying to connect: dns error: proto error: io error: No route to host (os error 148)" file = sled-agent/src/sled_agent.rs:1060 retry_after = 656.799206ms sled_id = 1e40a2ae-ffda-4a94-95f6-839f3349172b
3012023-12-06T17:40:52.682ZINFOSledAgent: contacting server nexus, registering sled baseboard = Baseboard { part_number: "i86pc", revision: 0, serial_number: "c6eef3d4b5d9454d80e3983e3692cc58" } file = sled-agent/src/sled_agent.rs:629 id = 1e40a2ae-ffda-4a94-95f6-839f3349172b sled_id = 1e40a2ae-ffda-4a94-95f6-839f3349172b
3022023-12-06T17:40:52.839ZINFOSledAgent: contacting server nexus, registering sled baseboard = Baseboard { part_number: "i86pc", revision: 0, serial_number: "c6eef3d4b5d9454d80e3983e3692cc58" } file = sled-agent/src/sled_agent.rs:629 id = 1e40a2ae-ffda-4a94-95f6-839f3349172b sled_id = 1e40a2ae-ffda-4a94-95f6-839f3349172b
3032023-12-06T17:40:53.123ZINFOSledAgent: contacting server nexus, registering sled baseboard = Baseboard { part_number: "i86pc", revision: 0, serial_number: "c6eef3d4b5d9454d80e3983e3692cc58" } file = sled-agent/src/sled_agent.rs:629 id = 1e40a2ae-ffda-4a94-95f6-839f3349172b sled_id = 1e40a2ae-ffda-4a94-95f6-839f3349172b
3042023-12-06T17:40:53.254ZINFOSledAgent: contacting server nexus, registering sled baseboard = Baseboard { part_number: "i86pc", revision: 0, serial_number: "c6eef3d4b5d9454d80e3983e3692cc58" } file = sled-agent/src/sled_agent.rs:629 id = 1e40a2ae-ffda-4a94-95f6-839f3349172b sled_id = 1e40a2ae-ffda-4a94-95f6-839f3349172b
3052023-12-06T17:40:53.317ZWARNSledAgent: failed to register as a metric producer with Nexus error = "Metric registration error: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/metrics/producers): error trying to connect: dns error: proto error: io error: No route to host (os error 148)" file = sled-agent/src/sled_agent.rs:1060 retry_after = 518.506625ms sled_id = 1e40a2ae-ffda-4a94-95f6-839f3349172b
3062023-12-06T17:40:53.680ZINFOSledAgent: contacting server nexus, registering sled baseboard = Baseboard { part_number: "i86pc", revision: 0, serial_number: "c6eef3d4b5d9454d80e3983e3692cc58" } file = sled-agent/src/sled_agent.rs:629 id = 1e40a2ae-ffda-4a94-95f6-839f3349172b sled_id = 1e40a2ae-ffda-4a94-95f6-839f3349172b
3072023-12-06T17:40:53.838ZWARNSledAgent: failed to register as a metric producer with Nexus error = "Metric registration error: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/metrics/producers): error trying to connect: dns error: proto error: io error: No route to host (os error 148)" file = sled-agent/src/sled_agent.rs:1060 retry_after = 2.19241492s sled_id = 1e40a2ae-ffda-4a94-95f6-839f3349172b
3082023-12-06T17:40:53.923ZINFOSledAgent: contacting server nexus, registering sled baseboard = Baseboard { part_number: "i86pc", revision: 0, serial_number: "c6eef3d4b5d9454d80e3983e3692cc58" } file = sled-agent/src/sled_agent.rs:629 id = 1e40a2ae-ffda-4a94-95f6-839f3349172b sled_id = 1e40a2ae-ffda-4a94-95f6-839f3349172b
3092023-12-06T17:40:54.283ZINFOSledAgent (ServiceManager): Zone booting file = illumos-utils/src/running_zone.rs:512 zone = oxz_internal_dns_00f82355-ab6a-44f8-adeb-55a0298e7676
3102023-12-06T17:40:54.376ZINFOSledAgent (ServiceManager): Zone booting file = illumos-utils/src/running_zone.rs:512 zone = oxz_internal_dns_be8d8c76-3f0a-4bd9-8466-47f0b72402ea
3112023-12-06T17:40:54.381ZINFOSledAgent (ServiceManager): Zone booting file = illumos-utils/src/running_zone.rs:512 zone = oxz_internal_dns_ac3d1180-e1c3-477d-a802-3f8bd92e2528
3122023-12-06T17:40:54.424ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_00f82355-ab6a-44f8-adeb-55a0298e7676") failed: Property not found. retry in 46.236501ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_00f82355-ab6a-44f8-adeb-55a0298e7676
3132023-12-06T17:40:54.479ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_00f82355-ab6a-44f8-adeb-55a0298e7676") failed: Property not found. retry in 82.518022ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_00f82355-ab6a-44f8-adeb-55a0298e7676
3142023-12-06T17:40:54.518ZINFOSledAgent: contacting server nexus, registering sled baseboard = Baseboard { part_number: "i86pc", revision: 0, serial_number: "c6eef3d4b5d9454d80e3983e3692cc58" } file = sled-agent/src/sled_agent.rs:629 id = 1e40a2ae-ffda-4a94-95f6-839f3349172b sled_id = 1e40a2ae-ffda-4a94-95f6-839f3349172b
3152023-12-06T17:40:54.586ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_be8d8c76-3f0a-4bd9-8466-47f0b72402ea") failed: Property not found. retry in 58.375118ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_be8d8c76-3f0a-4bd9-8466-47f0b72402ea
3162023-12-06T17:40:54.632ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_ac3d1180-e1c3-477d-a802-3f8bd92e2528") failed: Property not found. retry in 58.681917ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_ac3d1180-e1c3-477d-a802-3f8bd92e2528
3172023-12-06T17:40:54.725ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_00f82355-ab6a-44f8-adeb-55a0298e7676") failed: Property not found. retry in 169.433335ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_00f82355-ab6a-44f8-adeb-55a0298e7676
3182023-12-06T17:40:54.765ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {}
3192023-12-06T17:40:54.772ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_be8d8c76-3f0a-4bd9-8466-47f0b72402ea") failed: Property not found. retry in 96.686186ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_be8d8c76-3f0a-4bd9-8466-47f0b72402ea
3202023-12-06T17:40:54.792ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_ac3d1180-e1c3-477d-a802-3f8bd92e2528") failed: Property not found. retry in 88.362609ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_ac3d1180-e1c3-477d-a802-3f8bd92e2528
3212023-12-06T17:40:54.929ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_be8d8c76-3f0a-4bd9-8466-47f0b72402ea") failed: Property not found. retry in 166.49216ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_be8d8c76-3f0a-4bd9-8466-47f0b72402ea
3222023-12-06T17:40:54.930ZINFOSledAgent: contacting server nexus, registering sled baseboard = Baseboard { part_number: "i86pc", revision: 0, serial_number: "c6eef3d4b5d9454d80e3983e3692cc58" } file = sled-agent/src/sled_agent.rs:629 id = 1e40a2ae-ffda-4a94-95f6-839f3349172b sled_id = 1e40a2ae-ffda-4a94-95f6-839f3349172b
3232023-12-06T17:40:54.967ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_ac3d1180-e1c3-477d-a802-3f8bd92e2528") failed: Property not found. retry in 177.768868ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_ac3d1180-e1c3-477d-a802-3f8bd92e2528
3242023-12-06T17:40:54.978ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_00f82355-ab6a-44f8-adeb-55a0298e7676") failed: Property not found. retry in 232.385372ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_00f82355-ab6a-44f8-adeb-55a0298e7676
3252023-12-06T17:40:55.107ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_be8d8c76-3f0a-4bd9-8466-47f0b72402ea") failed: Property not found. retry in 332.44568ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_be8d8c76-3f0a-4bd9-8466-47f0b72402ea
3262023-12-06T17:40:55.153ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_ac3d1180-e1c3-477d-a802-3f8bd92e2528") failed: Property not found. retry in 598.058495ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_ac3d1180-e1c3-477d-a802-3f8bd92e2528
3272023-12-06T17:40:55.220ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_00f82355-ab6a-44f8-adeb-55a0298e7676") failed: Property not found. retry in 1.140088593s file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_00f82355-ab6a-44f8-adeb-55a0298e7676
3282023-12-06T17:40:55.449ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_be8d8c76-3f0a-4bd9-8466-47f0b72402ea") failed: Property not found. retry in 1.176388122s file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_be8d8c76-3f0a-4bd9-8466-47f0b72402ea
3292023-12-06T17:40:55.522ZINFOSledAgent: contacting server nexus, registering sled baseboard = Baseboard { part_number: "i86pc", revision: 0, serial_number: "c6eef3d4b5d9454d80e3983e3692cc58" } file = sled-agent/src/sled_agent.rs:629 id = 1e40a2ae-ffda-4a94-95f6-839f3349172b sled_id = 1e40a2ae-ffda-4a94-95f6-839f3349172b
3302023-12-06T17:40:55.541ZINFOSledAgent (ServiceManager): Ensuring bootstrap address fdb0:18c0:4d0d:a02a::2 exists in switch zone file = sled-agent/src/services.rs:1733
3312023-12-06T17:40:55.541ZINFOSledAgent (ServiceManager): Adding bootstrap address file = illumos-utils/src/running_zone.rs:625 zone = oxz_switch
3322023-12-06T17:40:55.764ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_ac3d1180-e1c3-477d-a802-3f8bd92e2528") failed: Property not found. retry in 552.820713ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_ac3d1180-e1c3-477d-a802-3f8bd92e2528
3332023-12-06T17:40:56.034ZWARNSledAgent: failed to register as a metric producer with Nexus error = "Metric registration error: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/metrics/producers): error trying to connect: dns error: proto error: io error: No route to host (os error 148)" file = sled-agent/src/sled_agent.rs:1060 retry_after = 5.662453817s sled_id = 1e40a2ae-ffda-4a94-95f6-839f3349172b
3342023-12-06T17:40:56.328ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_ac3d1180-e1c3-477d-a802-3f8bd92e2528") failed: Property not found. retry in 833.981717ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_ac3d1180-e1c3-477d-a802-3f8bd92e2528
3352023-12-06T17:40:56.369ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_00f82355-ab6a-44f8-adeb-55a0298e7676") failed: Property not found. retry in 537.569494ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_00f82355-ab6a-44f8-adeb-55a0298e7676
3362023-12-06T17:40:56.467ZINFOSledAgent: contacting server nexus, registering sled baseboard = Baseboard { part_number: "i86pc", revision: 0, serial_number: "c6eef3d4b5d9454d80e3983e3692cc58" } file = sled-agent/src/sled_agent.rs:629 id = 1e40a2ae-ffda-4a94-95f6-839f3349172b sled_id = 1e40a2ae-ffda-4a94-95f6-839f3349172b
3372023-12-06T17:40:56.634ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_be8d8c76-3f0a-4bd9-8466-47f0b72402ea") failed: Property not found. retry in 738.799936ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_be8d8c76-3f0a-4bd9-8466-47f0b72402ea
3382023-12-06T17:40:56.916ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_00f82355-ab6a-44f8-adeb-55a0298e7676") failed: Property not found. retry in 627.635978ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_00f82355-ab6a-44f8-adeb-55a0298e7676
3392023-12-06T17:40:57.170ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_ac3d1180-e1c3-477d-a802-3f8bd92e2528") failed: Property not found. retry in 1.25502771s file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_ac3d1180-e1c3-477d-a802-3f8bd92e2528
3402023-12-06T17:40:57.205ZINFOSledAgent: contacting server nexus, registering sled baseboard = Baseboard { part_number: "i86pc", revision: 0, serial_number: "c6eef3d4b5d9454d80e3983e3692cc58" } file = sled-agent/src/sled_agent.rs:629 id = 1e40a2ae-ffda-4a94-95f6-839f3349172b sled_id = 1e40a2ae-ffda-4a94-95f6-839f3349172b
3412023-12-06T17:40:57.381ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_be8d8c76-3f0a-4bd9-8466-47f0b72402ea") failed: Property not found. retry in 902.149335ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_be8d8c76-3f0a-4bd9-8466-47f0b72402ea
3422023-12-06T17:40:57.553ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_00f82355-ab6a-44f8-adeb-55a0298e7676") failed: Property not found. retry in 743.095443ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_00f82355-ab6a-44f8-adeb-55a0298e7676
3432023-12-06T17:40:58.293ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_be8d8c76-3f0a-4bd9-8466-47f0b72402ea") failed: Property not found. retry in 1.491888532s file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_be8d8c76-3f0a-4bd9-8466-47f0b72402ea
3442023-12-06T17:40:58.306ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_00f82355-ab6a-44f8-adeb-55a0298e7676") failed: Property not found. retry in 1.184267469s file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_00f82355-ab6a-44f8-adeb-55a0298e7676
3452023-12-06T17:40:58.434ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_ac3d1180-e1c3-477d-a802-3f8bd92e2528") failed: Property not found. retry in 821.305413ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_ac3d1180-e1c3-477d-a802-3f8bd92e2528
3462023-12-06T17:40:58.565ZINFOSledAgent: contacting server nexus, registering sled baseboard = Baseboard { part_number: "i86pc", revision: 0, serial_number: "c6eef3d4b5d9454d80e3983e3692cc58" } file = sled-agent/src/sled_agent.rs:629 id = 1e40a2ae-ffda-4a94-95f6-839f3349172b sled_id = 1e40a2ae-ffda-4a94-95f6-839f3349172b
3472023-12-06T17:40:59.302ZINFOSledAgent (ServiceManager): Ensuring address fd00:1122:3344:3::1 exists file = sled-agent/src/services.rs:1769
3482023-12-06T17:40:59.302ZINFOSledAgent (ServiceManager): Adding address: Static(V6(Ipv6Network { addr: fd00:1122:3344:3::1, prefix: 64 })) file = illumos-utils/src/running_zone.rs:608 zone = oxz_internal_dns_ac3d1180-e1c3-477d-a802-3f8bd92e2528
3492023-12-06T17:40:59.869ZINFOSledAgent (ServiceManager): Ensuring address fd00:1122:3344:3::1 exists - OK file = sled-agent/src/services.rs:1777
3502023-12-06T17:40:59.869ZINFOSledAgent (ServiceManager): Zone not using gateway (even though underlay is up) file = sled-agent/src/services.rs:1801
3512023-12-06T17:41:00.059ZINFOSledAgent (ServiceManager): Setting up internal-dns service file = sled-agent/src/services.rs:1995
3522023-12-06T17:41:00.078ZINFOSledAgent (ServiceManager): Forwarding bootstrap traffic via oxBootstrap0 to fe80::8:20ff:fe68:5974 file = sled-agent/src/services.rs:1740
3532023-12-06T17:41:00.083ZINFOSledAgent (ServiceManager): Zone not using gateway (even though underlay is up) file = sled-agent/src/services.rs:1801
3542023-12-06T17:41:00.201ZINFOSledAgent (ServiceManager): Setting up dendrite service file = sled-agent/src/services.rs:2280
3552023-12-06T17:41:00.442ZINFOSledAgent (ServiceManager): Setting up MGS service file = sled-agent/src/services.rs:2159
3562023-12-06T17:41:00.589ZINFOSledAgent (BootstrapAgentStartup): Sending prefix to ddmd for advertisement DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:91 prefix = Ipv6Prefix { addr: fd00:1122:3344:3::, len: 64 }
3572023-12-06T17:41:00.590ZINFOSledAgent: contacting server nexus, registering sled baseboard = Baseboard { part_number: "i86pc", revision: 0, serial_number: "c6eef3d4b5d9454d80e3983e3692cc58" } file = sled-agent/src/sled_agent.rs:629 id = 1e40a2ae-ffda-4a94-95f6-839f3349172b sled_id = 1e40a2ae-ffda-4a94-95f6-839f3349172b
3582023-12-06T17:41:00.593ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {}
3592023-12-06T17:41:00.640ZINFOSledAgent (ServiceManager): Ensuring address fd00:1122:3344:2::1 exists file = sled-agent/src/services.rs:1769
3602023-12-06T17:41:00.640ZINFOSledAgent (ServiceManager): Adding address: Static(V6(Ipv6Network { addr: fd00:1122:3344:2::1, prefix: 64 })) file = illumos-utils/src/running_zone.rs:608 zone = oxz_internal_dns_be8d8c76-3f0a-4bd9-8466-47f0b72402ea
3612023-12-06T17:41:00.747ZINFOSledAgent (ServiceManager): Setting up wicketd service file = sled-agent/src/services.rs:2194
3622023-12-06T17:41:00.943ZINFOSledAgent (ServiceManager): Setting up mgd service file = sled-agent/src/services.rs:2461
3632023-12-06T17:41:01.087ZINFOSledAgent (ServiceManager): Setting up mg-ddm service file = sled-agent/src/services.rs:2466
3642023-12-06T17:41:01.193ZINFOSledAgent (ServiceManager): Ensuring address fd00:1122:3344:2::1 exists - OK file = sled-agent/src/services.rs:1777
3652023-12-06T17:41:01.193ZINFOSledAgent (ServiceManager): Zone not using gateway (even though underlay is up) file = sled-agent/src/services.rs:1801
3662023-12-06T17:41:01.268ZINFOSledAgent (ServiceManager): Setting up tfport service file = sled-agent/src/services.rs:2400
3672023-12-06T17:41:01.372ZINFOSledAgent (ServiceManager): Setting up internal-dns service file = sled-agent/src/services.rs:1995
3682023-12-06T17:41:01.437ZINFOSledAgent (ServiceManager): Setting up Simulated SP service file = sled-agent/src/services.rs:2188
3692023-12-06T17:41:01.446ZINFOSledAgent (ServiceManager): Re-enabling running switch zone (new address) file = sled-agent/src/services.rs:3294 new = [fd00:1122:3344:101::2, ::1] old = [::1]
3702023-12-06T17:41:01.446ZINFOSledAgent (ServiceManager): Ensuring address fd00:1122:3344:101::2 exists file = sled-agent/src/services.rs:3310
3712023-12-06T17:41:01.446ZINFOSledAgent (ServiceManager): Adding address: Static(V6(Ipv6Network { addr: fd00:1122:3344:101::2, prefix: 64 })) file = illumos-utils/src/running_zone.rs:608 zone = oxz_switch
3722023-12-06T17:41:01.921ZINFOSledAgent (ServiceManager): Ensuring address fd00:1122:3344:1::1 exists file = sled-agent/src/services.rs:1769
3732023-12-06T17:41:01.921ZINFOSledAgent (ServiceManager): Adding address: Static(V6(Ipv6Network { addr: fd00:1122:3344:1::1, prefix: 64 })) file = illumos-utils/src/running_zone.rs:608 zone = oxz_internal_dns_00f82355-ab6a-44f8-adeb-55a0298e7676
3742023-12-06T17:41:01.996ZINFOSledAgent (ServiceManager): Ensuring address fd00:1122:3344:101::2 exists - OK file = sled-agent/src/services.rs:3318
3752023-12-06T17:41:02.002ZINFOSledAgent (ServiceManager): configuring dendrite zone file = sled-agent/src/services.rs:3374
3762023-12-06T17:41:02.184ZINFOSledAgent (ServiceManager): configuring wicketd file = sled-agent/src/services.rs:3414 rack_subnet = fd00:1122:3344::
3772023-12-06T17:41:02.236ZINFOSledAgent (ServiceManager): Determining physical location of our switch zone at fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:359
3782023-12-06T17:41:02.269ZWARNSledAgent (ServiceManager): Failed to get switch ID from MGS (retrying in 43.682146ms) error = Error Response: status: 503 Service Unavailable; headers: {"content-type": "application/json", "x-request-id": "3293e33c-d182-4558-b7a4-4583976a4e8a", "content-length": "152", "date": "Wed, 06 Dec 2023 17:41:01 GMT"}; value: Error { error_code: Some("DiscoveryNotYetComplete"), message: "discovery process not yet complete", request_id: "3293e33c-d182-4558-b7a4-4583976a4e8a" } file = sled-agent/src/bootstrap/early_networking.rs:378
3792023-12-06T17:41:02.314ZWARNSledAgent (ServiceManager): Failed to get switch ID from MGS (retrying in 66.537065ms) error = Error Response: status: 503 Service Unavailable; headers: {"content-type": "application/json", "x-request-id": "654248e0-d795-49d7-b581-79320a309b8a", "content-length": "152", "date": "Wed, 06 Dec 2023 17:41:01 GMT"}; value: Error { error_code: Some("DiscoveryNotYetComplete"), message: "discovery process not yet complete", request_id: "654248e0-d795-49d7-b581-79320a309b8a" } file = sled-agent/src/bootstrap/early_networking.rs:378
3802023-12-06T17:41:02.382ZWARNSledAgent (ServiceManager): Failed to get switch ID from MGS (retrying in 150.964841ms) error = Error Response: status: 503 Service Unavailable; headers: {"content-type": "application/json", "x-request-id": "ac4b25f4-c6a2-4ade-b787-f096b1d4f633", "content-length": "152", "date": "Wed, 06 Dec 2023 17:41:01 GMT"}; value: Error { error_code: Some("DiscoveryNotYetComplete"), message: "discovery process not yet complete", request_id: "ac4b25f4-c6a2-4ade-b787-f096b1d4f633" } file = sled-agent/src/bootstrap/early_networking.rs:378
3812023-12-06T17:41:02.472ZINFOSledAgent (ServiceManager): Ensuring address fd00:1122:3344:1::1 exists - OK file = sled-agent/src/services.rs:1777
3822023-12-06T17:41:02.472ZINFOSledAgent (ServiceManager): Zone not using gateway (even though underlay is up) file = sled-agent/src/services.rs:1801
3832023-12-06T17:41:02.535ZINFOSledAgent (ServiceManager): Initializing 1 Uplinks on Switch0 at fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:407
3842023-12-06T17:41:02.567ZINFOSledAgent (ServiceManager): Building Port Configuration file = sled-agent/src/bootstrap/early_networking.rs:533
3852023-12-06T17:41:02.567ZINFOSledAgent (ServiceManager): Checking dendrite uptime file = sled-agent/src/bootstrap/early_networking.rs:602
3862023-12-06T17:41:02.568ZINFOSledAgent (ServiceManager): Dendrite online file = sled-agent/src/bootstrap/early_networking.rs:605 uptime = 2
3872023-12-06T17:41:02.568ZINFOSledAgent (ServiceManager): Configuring boundary services loopback address on switch config = Ipv6Entry {\n addr: fd00:99::1,\n tag: "omicron",\n} file = sled-agent/src/bootstrap/early_networking.rs:429
3882023-12-06T17:41:02.568ZINFOSledAgent (ServiceManager): Configuring default uplink on switch config = PortSettings {\n links: {\n "0": LinkSettings {\n addrs: [\n 10.151.2.174,\n ],\n params: LinkCreate {\n autoneg: false,\n fec: None,\n kr: false,\n lane: Some(\n LinkId(\n 0,\n ),\n ),\n speed: Speed40G,\n },\n },\n },\n v4_routes: {\n "0.0.0.0/0": [\n RouteSettingsV4 {\n link_id: 0,\n nexthop: 10.151.2.100,\n },\n ],\n },\n v6_routes: {},\n} file = sled-agent/src/bootstrap/early_networking.rs:440
3892023-12-06T17:41:02.569ZINFOSledAgent (ServiceManager): advertising boundary services loopback address file = sled-agent/src/bootstrap/early_networking.rs:457
3902023-12-06T17:41:02.634ZINFOSledAgent (ServiceManager): Sending prefix to ddmd for advertisement DdmAdminClient = [fd00:1122:3344:101::2]:8000 file = clients/ddm-admin-client/src/lib.rs:91 prefix = Ipv6Prefix { addr: fd00:99::, len: 64 }
3912023-12-06T17:41:02.646ZINFOSledAgent (ServiceManager): Setting up internal-dns service file = sled-agent/src/services.rs:1995
3922023-12-06T17:41:02.680ZINFOSledAgent (StorageManager): Received DisksChanged({}) file = sled-storage/src/manager.rs:346
3932023-12-06T17:41:03.127ZINFOSledAgent (BootstrapAgentStartup): Sending prefix to ddmd for advertisement DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:91 prefix = Ipv6Prefix { addr: fd00:1122:3344:2::, len: 64 }
3942023-12-06T17:41:03.179ZINFOSledAgent (ServiceManager): Writing ledger to /pool/int/a462a7f7-b628-40fe-80ff-4e4189e2d62b/config/.omicron-zones.json.tmp file = common/src/ledger.rs:198
3952023-12-06T17:41:03.179ZINFOSledAgent (BootstrapAgentStartup): Sending prefix to ddmd for advertisement DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:91 prefix = Ipv6Prefix { addr: fd00:1122:3344:1::, len: 64 }
3962023-12-06T17:41:03.179ZINFOSledAgent (ServiceManager): Writing ledger to /pool/int/b462a7f7-b628-40fe-80ff-4e4189e2d62b/config/.omicron-zones.json.tmp file = common/src/ledger.rs:198
3972023-12-06T17:41:03.180ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 10778923 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::1]:54836 req_id = 18181feb-5ea4-4166-a2c3-f1c1d9902ca3 response_code = 204 uri = /omicron-zones
3982023-12-06T17:41:03.181ZINFOSledAgent (RSS): Configuring DNS server dns_config_addr = [fd00:1122:3344:1::1]:5353 file = sled-agent/src/rack_setup/service.rs:417
3992023-12-06T17:41:04.484ZINFOSledAgent (RSS): Configuring DNS server dns_config_addr = [fd00:1122:3344:2::1]:5353 file = sled-agent/src/rack_setup/service.rs:417
4002023-12-06T17:41:04.524ZINFOSledAgent (RSS): Configuring DNS server dns_config_addr = [fd00:1122:3344:3::1]:5353 file = sled-agent/src/rack_setup/service.rs:417
4012023-12-06T17:41:04.561ZINFOSledAgent (RSS): Configured all DNS servers file = sled-agent/src/rack_setup/service.rs:453
4022023-12-06T17:41:04.561ZINFOSledAgent (RSS): Resolving switch zone addresses in DNS file = sled-agent/src/bootstrap/early_networking.rs:253
4032023-12-06T17:41:04.593ZINFOSledAgent (RSS): Querying MGS to determine switch location addr = fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:270
4042023-12-06T17:41:04.594ZINFOSledAgent (RSS): Found Switch0 at fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:295
4052023-12-06T17:41:04.594ZINFOSledAgent (RSS): Successfully looked up all expected switch zone underlay addresses addrs = {Switch0: fd00:1122:3344:101::2} file = sled-agent/src/bootstrap/early_networking.rs:187
4062023-12-06T17:41:04.623ZINFOSledAgent (RSS): attempting to set up sled's Omicron zones: OmicronZonesConfig { generation: Generation(3), zones: [OmicronZoneConfig { id: 00f82355-ab6a-44f8-adeb-55a0298e7676, underlay_address: fd00:1122:3344:1::1, zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 14b4dc87-ab46-49fb-a4b4-d361ae214c03, kind: External } }, http_address: [fd00:1122:3344:1::1]:5353, dns_address: [fd00:1122:3344:1::1]:53, gz_address: fd00:1122:3344:1::2, gz_address_index: 0 } }, OmicronZoneConfig { id: be8d8c76-3f0a-4bd9-8466-47f0b72402ea, underlay_address: fd00:1122:3344:2::1, zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 24b4dc87-ab46-49fb-a4b4-d361ae214c03, kind: External } }, http_address: [fd00:1122:3344:2::1]:5353, dns_address: [fd00:1122:3344:2::1]:53, gz_address: fd00:1122:3344:2::2, gz_address_index: 1 } }, OmicronZoneConfig { id: ac3d1180-e1c3-477d-a802-3f8bd92e2528, underlay_address: fd00:1122:3344:3::1, zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 31bd71cd-4736-4a12-a387-9b74b050396f, kind: External } }, http_address: [fd00:1122:3344:3::1]:5353, dns_address: [fd00:1122:3344:3::1]:53, gz_address: fd00:1122:3344:3::2, gz_address_index: 2 } }, OmicronZoneConfig { id: 0e075832-9eaa-40c5-a328-dcde3812ee47, underlay_address: fd00:1122:3344:101::1b, zone_type: BoundaryNtp { address: [fd00:1122:3344:101::1b]:123, ntp_servers: ["0.pool.ntp.org"], dns_servers: [1.1.1.1, 9.9.9.9], domain: None, nic: NetworkInterface { id: 1277989f-b18b-4c6b-8166-0b015ebc20df, kind: Service { id: 0e075832-9eaa-40c5-a328-dcde3812ee47 }, name: Name("ntp-0e075832-9eaa-40c5-a328-dcde3812ee47"), ip: 172.30.3.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 143, 64])), subnet: V4(Ipv4Net(Ipv4Network { addr: 172.30.3.0, prefix: 24 })), vni: Vni(100), primary: true, slot: 0 }, snat_cfg: SourceNatConfig { ip: 10.151.2.169, first_port: 0, last_port: 16383 } } }] } file = sled-agent/src/rack_setup/service.rs:299 sled_address = [fd00:1122:3344:101::1]:12345
4072023-12-06T17:41:04.623ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:769 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:34028
4082023-12-06T17:41:04.624ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 00f82355-ab6a-44f8-adeb-55a0298e7676, dataset_name: DatasetName { pool_name: ZpoolName { id: 14b4dc87-ab46-49fb-a4b4-d361ae214c03, kind: External }, kind: InternalDns }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } }) file = sled-storage/src/manager.rs:346
4092023-12-06T17:41:04.624ZINFOSledAgent (StorageManager): add_dataset: NewFilesystemRequest { dataset_id: 00f82355-ab6a-44f8-adeb-55a0298e7676, dataset_name: DatasetName { pool_name: ZpoolName { id: 14b4dc87-ab46-49fb-a4b4-d361ae214c03, kind: External }, kind: InternalDns }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } } file = sled-storage/src/manager.rs:564
4102023-12-06T17:41:04.645ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: be8d8c76-3f0a-4bd9-8466-47f0b72402ea, dataset_name: DatasetName { pool_name: ZpoolName { id: 24b4dc87-ab46-49fb-a4b4-d361ae214c03, kind: External }, kind: InternalDns }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } }) file = sled-storage/src/manager.rs:346
4112023-12-06T17:41:04.645ZINFOSledAgent (StorageManager): add_dataset: NewFilesystemRequest { dataset_id: be8d8c76-3f0a-4bd9-8466-47f0b72402ea, dataset_name: DatasetName { pool_name: ZpoolName { id: 24b4dc87-ab46-49fb-a4b4-d361ae214c03, kind: External }, kind: InternalDns }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } } file = sled-storage/src/manager.rs:564
4122023-12-06T17:41:04.664ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: ac3d1180-e1c3-477d-a802-3f8bd92e2528, dataset_name: DatasetName { pool_name: ZpoolName { id: 31bd71cd-4736-4a12-a387-9b74b050396f, kind: External }, kind: InternalDns }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } }) file = sled-storage/src/manager.rs:346
4132023-12-06T17:41:04.666ZINFOSledAgent (StorageManager): add_dataset: NewFilesystemRequest { dataset_id: ac3d1180-e1c3-477d-a802-3f8bd92e2528, dataset_name: DatasetName { pool_name: ZpoolName { id: 31bd71cd-4736-4a12-a387-9b74b050396f, kind: External }, kind: InternalDns }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } } file = sled-storage/src/manager.rs:564
4142023-12-06T17:41:04.682ZINFOSledAgent (StorageManager): Received GetLatestResources(Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) }) file = sled-storage/src/manager.rs:346
4152023-12-06T17:41:04.682ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/a462a7f7-b628-40fe-80ff-4e4189e2d62b/config/omicron-zones.json file = common/src/ledger.rs:176
4162023-12-06T17:41:04.683ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/b462a7f7-b628-40fe-80ff-4e4189e2d62b/config/omicron-zones.json file = common/src/ledger.rs:176
4172023-12-06T17:41:04.683ZINFOSledAgent (StorageManager): Received GetLatestResources(Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) }) file = sled-storage/src/manager.rs:346
4182023-12-06T17:41:04.685ZINFOSledAgent (ServiceManager): Resolving switch zone addresses in DNS file = sled-agent/src/bootstrap/early_networking.rs:253
4192023-12-06T17:41:04.717ZINFOSledAgent (ServiceManager): Querying MGS to determine switch location addr = fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:270
4202023-12-06T17:41:04.718ZINFOSledAgent (ServiceManager): Found Switch0 at fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:295
4212023-12-06T17:41:04.718ZINFOSledAgent (ServiceManager): Successfully looked up all expected switch zone underlay addresses addrs = {Switch0: fd00:1122:3344:101::2} file = sled-agent/src/bootstrap/early_networking.rs:187
4222023-12-06T17:41:04.764ZINFOSledAgent (PortManager): Created OPTE port file = illumos-utils/src/opte/port_manager.rs:383 port = Port { inner: PortInner { name: "opte0", _ip: 172.30.3.5, mac: MacAddr6([168, 64, 37, 255, 143, 64]), slot: 0, vni: Vni { inner: 100 }, gateway: Gateway { mac: MacAddr6([168, 64, 37, 255, 119, 119]), ip: 172.30.3.1 }, vnic: "vopte0" } }
4232023-12-06T17:41:04.764ZINFOSledAgent (ServiceManager): creating NAT entry for service file = sled-agent/src/services.rs:1226 zone_type = ntp
4242023-12-06T17:41:04.764ZINFOSledAgent (ServiceManager): no nat entry found for: V4( Ipv4Network { addr: 10.151.2.169, prefix: 32, }, ) file = clients/dpd-client/src/lib.rs:98
4252023-12-06T17:41:04.764ZINFOSledAgent (ServiceManager): creating nat entry for: V4( Ipv4Network { addr: 10.151.2.169, prefix: 32, }, ) file = clients/dpd-client/src/lib.rs:105
4262023-12-06T17:41:04.765ZINFOSledAgent (ServiceManager): creation of nat entry successful for: V4( Ipv4Network { addr: 10.151.2.169, prefix: 32, }, ) file = clients/dpd-client/src/lib.rs:133
4272023-12-06T17:41:04.767ZINFOSledAgent (StorageManager): Received GetLatestResources(Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) }) file = sled-storage/src/manager.rs:346
4282023-12-06T17:41:04.790ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_ntp_0e075832-9eaa-40c5-a328-dcde3812ee47 file = illumos-utils/src/zone.rs:314
4292023-12-06T17:41:04.808ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_ntp_0e075832-9eaa-40c5-a328-dcde3812ee47 file = illumos-utils/src/zone.rs:353
4302023-12-06T17:41:05.596ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::60fc:8ff:fe95:bcc8": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
4312023-12-06T17:41:05.758ZINFOSledAgent (ServiceManager): Zone booting file = illumos-utils/src/running_zone.rs:512 zone = oxz_ntp_0e075832-9eaa-40c5-a328-dcde3812ee47
4322023-12-06T17:41:05.893ZWARNSledAgent (ServiceManager): wait for service Some("oxz_ntp_0e075832-9eaa-40c5-a328-dcde3812ee47") failed: Property not found. retry in 58.626742ms file = illumos-utils/src/svc.rs:36 zone = oxz_ntp_0e075832-9eaa-40c5-a328-dcde3812ee47
4332023-12-06T17:41:05.962ZWARNSledAgent (ServiceManager): wait for service Some("oxz_ntp_0e075832-9eaa-40c5-a328-dcde3812ee47") failed: Property not found. retry in 74.881479ms file = illumos-utils/src/svc.rs:36 zone = oxz_ntp_0e075832-9eaa-40c5-a328-dcde3812ee47
4342023-12-06T17:41:06.068ZWARNSledAgent (ServiceManager): wait for service Some("oxz_ntp_0e075832-9eaa-40c5-a328-dcde3812ee47") failed: Property not found. retry in 262.933583ms file = illumos-utils/src/svc.rs:36 zone = oxz_ntp_0e075832-9eaa-40c5-a328-dcde3812ee47
4352023-12-06T17:41:06.117ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:769 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:45679
4362023-12-06T17:41:06.117ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 78 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:45679 req_id = 01bec47c-1ed9-43e4-8964-542004a170c3 response_code = 200 uri = /baseboard
4372023-12-06T17:41:06.343ZWARNSledAgent (ServiceManager): wait for service Some("oxz_ntp_0e075832-9eaa-40c5-a328-dcde3812ee47") failed: Property not found. retry in 337.177037ms file = illumos-utils/src/svc.rs:36 zone = oxz_ntp_0e075832-9eaa-40c5-a328-dcde3812ee47
4382023-12-06T17:41:06.689ZWARNSledAgent (ServiceManager): wait for service Some("oxz_ntp_0e075832-9eaa-40c5-a328-dcde3812ee47") failed: Property not found. retry in 670.689792ms file = illumos-utils/src/svc.rs:36 zone = oxz_ntp_0e075832-9eaa-40c5-a328-dcde3812ee47
4392023-12-06T17:41:06.700ZWARNSledAgent: failed to register as a metric producer with Nexus error = "Metric registration error: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/metrics/producers): error trying to connect: dns error: no record found for Query { name: Name(\\"_nexus._tcp.control-plane.oxide.internal.\\"), query_type: SRV, query_class: IN }" file = sled-agent/src/sled_agent.rs:1060 retry_after = 10.095111057s sled_id = 1e40a2ae-ffda-4a94-95f6-839f3349172b
4402023-12-06T17:41:07.372ZWARNSledAgent (ServiceManager): wait for service Some("oxz_ntp_0e075832-9eaa-40c5-a328-dcde3812ee47") failed: Property not found. retry in 552.952919ms file = illumos-utils/src/svc.rs:36 zone = oxz_ntp_0e075832-9eaa-40c5-a328-dcde3812ee47
4412023-12-06T17:41:07.934ZWARNSledAgent (ServiceManager): wait for service Some("oxz_ntp_0e075832-9eaa-40c5-a328-dcde3812ee47") failed: Property not found. retry in 830.246311ms file = illumos-utils/src/svc.rs:36 zone = oxz_ntp_0e075832-9eaa-40c5-a328-dcde3812ee47
4422023-12-06T17:41:08.774ZWARNSledAgent (ServiceManager): wait for service Some("oxz_ntp_0e075832-9eaa-40c5-a328-dcde3812ee47") failed: Property not found. retry in 701.254334ms file = illumos-utils/src/svc.rs:36 zone = oxz_ntp_0e075832-9eaa-40c5-a328-dcde3812ee47
4432023-12-06T17:41:09.486ZWARNSledAgent (ServiceManager): wait for service Some("oxz_ntp_0e075832-9eaa-40c5-a328-dcde3812ee47") failed: Property not found. retry in 711.9007ms file = illumos-utils/src/svc.rs:36 zone = oxz_ntp_0e075832-9eaa-40c5-a328-dcde3812ee47
4442023-12-06T17:41:10.208ZWARNSledAgent (ServiceManager): wait for service Some("oxz_ntp_0e075832-9eaa-40c5-a328-dcde3812ee47") failed: Property not found. retry in 1.498116088s file = illumos-utils/src/svc.rs:36 zone = oxz_ntp_0e075832-9eaa-40c5-a328-dcde3812ee47
4452023-12-06T17:41:10.597ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::60fc:8ff:fe95:bcc8": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
4462023-12-06T17:41:11.748ZINFOSledAgent (ServiceManager): Ensuring address fd00:1122:3344:101::1b exists file = sled-agent/src/services.rs:1769
4472023-12-06T17:41:11.748ZINFOSledAgent (ServiceManager): Adding address: Static(V6(Ipv6Network { addr: fd00:1122:3344:101::1b, prefix: 64 })) file = illumos-utils/src/running_zone.rs:608 zone = oxz_ntp_0e075832-9eaa-40c5-a328-dcde3812ee47
4482023-12-06T17:41:12.297ZINFOSledAgent (ServiceManager): Ensuring address fd00:1122:3344:101::1b exists - OK file = sled-agent/src/services.rs:1777
4492023-12-06T17:41:12.297ZINFOSledAgent (ServiceManager): Zone using sled underlay as gateway file = sled-agent/src/services.rs:1796
4502023-12-06T17:41:12.498ZINFOSledAgent (ServiceManager): Set up NTP service boundary=true, Servers=["0.pool.ntp.org"] file = sled-agent/src/services.rs:2086
4512023-12-06T17:41:12.524ZINFOSledAgent (ServiceManager): Ensuring address for OPTE port file = illumos-utils/src/running_zone.rs:652 zone = oxz_ntp_0e075832-9eaa-40c5-a328-dcde3812ee47
4522023-12-06T17:41:18.922ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::60fc:8ff:fe95:bcc8": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
4532023-12-06T17:41:18.970ZINFOSledAgent (ServiceManager): Writing ledger to /pool/int/a462a7f7-b628-40fe-80ff-4e4189e2d62b/config/.omicron-zones.json.tmp file = common/src/ledger.rs:198
4542023-12-06T17:41:18.971ZINFOSledAgent (ServiceManager): Writing ledger to /pool/int/b462a7f7-b628-40fe-80ff-4e4189e2d62b/config/.omicron-zones.json.tmp file = common/src/ledger.rs:198
4552023-12-06T17:41:18.971ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 14347567 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::1]:34028 req_id = 3c04802b-edf3-46ec-bcaa-855246d159c8 response_code = 204 uri = /omicron-zones
4562023-12-06T17:41:18.974ZINFOSledAgent (RSS): Waiting for rack time synchronization file = sled-agent/src/rack_setup/service.rs:494
4572023-12-06T17:41:19.004ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:474
4582023-12-06T17:41:19.004ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:769 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:43073
4592023-12-06T17:41:19.014ZINFOSledAgent (ServiceManager): chronyc command failed: Error running command in zone 'oxz_ntp_0e075832-9eaa-40c5-a328-dcde3812ee47': Command [/usr/bin/chronyc -c tracking] executed and failed with status: exit status: 1 stdout: 506 Cannot talk to daemon stderr: file = sled-agent/src/services.rs:3028
4602023-12-06T17:41:19.014ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = NTP zone not ready file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:841 latency_us = 10069 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:43073 req_id = 6e7f5824-c9ff-4ad3-904b-fea56c851ac1 response_code = 500 uri = /timesync
4612023-12-06T17:41:19.018ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:524
4622023-12-06T17:41:19.109ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:474
4632023-12-06T17:41:19.109ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:769 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:54962
4642023-12-06T17:41:19.118ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 8947 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:54962 req_id = 3e937a00-a82c-40d2-9bac-c355e3944da5 response_code = 200 uri = /timesync
4652023-12-06T17:41:19.122ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 2139029761, ip_addr: ::, stratum: 10, ref_time: 1701884477.750389, correction: 0.0 } file = sled-agent/src/rack_setup/service.rs:502
4662023-12-06T17:41:19.122ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:524
4672023-12-06T17:41:19.232ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:474
4682023-12-06T17:41:19.232ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:769 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:49441
4692023-12-06T17:41:19.241ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 8518 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:49441 req_id = c07cb095-4079-45b5-9003-56207c1e57be response_code = 200 uri = /timesync
4702023-12-06T17:41:19.243ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 2139029761, ip_addr: ::, stratum: 10, ref_time: 1701884477.750389, correction: 0.0 } file = sled-agent/src/rack_setup/service.rs:502
4712023-12-06T17:41:19.243ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:524
4722023-12-06T17:41:19.486ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:474
4732023-12-06T17:41:19.486ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:769 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:60062
4742023-12-06T17:41:19.494ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 8193 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:60062 req_id = 9798547f-9ab5-467c-a2b3-ecaad68fa86d response_code = 200 uri = /timesync
4752023-12-06T17:41:19.496ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 2139029761, ip_addr: ::, stratum: 10, ref_time: 1701884477.750389, correction: 0.0 } file = sled-agent/src/rack_setup/service.rs:502
4762023-12-06T17:41:19.496ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:524
4772023-12-06T17:41:19.777ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:474
4782023-12-06T17:41:19.777ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:769 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:46196
4792023-12-06T17:41:19.787ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 9088 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:46196 req_id = 4d14803e-d3fd-45d7-b41d-e622378e2b2c response_code = 200 uri = /timesync
4802023-12-06T17:41:19.789ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 2139029761, ip_addr: ::, stratum: 10, ref_time: 1701884477.750389, correction: 0.0 } file = sled-agent/src/rack_setup/service.rs:502
4812023-12-06T17:41:19.789ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:524
4822023-12-06T17:41:19.997ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:474
4832023-12-06T17:41:19.997ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:769 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:50145
4842023-12-06T17:41:20.005ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 7848 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:50145 req_id = 876e9a34-31e8-4110-802c-f3e49647a63a response_code = 200 uri = /timesync
4852023-12-06T17:41:20.008ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 2139029761, ip_addr: ::, stratum: 10, ref_time: 1701884477.750389, correction: 0.0 } file = sled-agent/src/rack_setup/service.rs:502
4862023-12-06T17:41:20.008ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:524
4872023-12-06T17:41:20.380ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:474
4882023-12-06T17:41:20.380ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:769 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:56828
4892023-12-06T17:41:20.388ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 8186 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:56828 req_id = 76b669de-6193-4b7e-8bcf-f5f934f14550 response_code = 200 uri = /timesync
4902023-12-06T17:41:20.390ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 2139029761, ip_addr: ::, stratum: 10, ref_time: 1701884477.750389, correction: 0.0 } file = sled-agent/src/rack_setup/service.rs:502
4912023-12-06T17:41:20.390ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:524
4922023-12-06T17:41:20.593ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:474
4932023-12-06T17:41:20.593ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:769 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:60296
4942023-12-06T17:41:20.601ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 8269 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:60296 req_id = 10de6085-6354-4f2a-aa0a-af25efa70594 response_code = 200 uri = /timesync
4952023-12-06T17:41:20.604ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 2139029761, ip_addr: ::, stratum: 10, ref_time: 1701884477.750389, correction: 0.0 } file = sled-agent/src/rack_setup/service.rs:502
4962023-12-06T17:41:20.604ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:524
4972023-12-06T17:41:21.096ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:474
4982023-12-06T17:41:21.096ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:769 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:43469
4992023-12-06T17:41:21.104ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 8208 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:43469 req_id = b7e8f6a9-d4ba-409c-9b7c-5467fade7bd9 response_code = 200 uri = /timesync
5002023-12-06T17:41:21.106ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 2139029761, ip_addr: ::, stratum: 10, ref_time: 1701884477.750389, correction: 0.0 } file = sled-agent/src/rack_setup/service.rs:502
5012023-12-06T17:41:21.106ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:524
5022023-12-06T17:41:21.555ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:474
5032023-12-06T17:41:21.556ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:769 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:34816
5042023-12-06T17:41:21.564ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 8677 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:34816 req_id = 7e4c9cdd-ca9b-42be-b0c4-ea1fdeafa0b6 response_code = 200 uri = /timesync
5052023-12-06T17:41:21.566ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 2139029761, ip_addr: ::, stratum: 10, ref_time: 1701884477.750389, correction: 0.0 } file = sled-agent/src/rack_setup/service.rs:502
5062023-12-06T17:41:21.566ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:524
5072023-12-06T17:41:22.091ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:474
5082023-12-06T17:41:22.092ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:769 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:57633
5092023-12-06T17:41:22.100ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 8457 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:57633 req_id = e54b0a98-7c6d-4f5b-9465-632c6cf6b172 response_code = 200 uri = /timesync
5102023-12-06T17:41:22.102ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 2139029761, ip_addr: ::, stratum: 10, ref_time: 1701884477.750389, correction: 0.0 } file = sled-agent/src/rack_setup/service.rs:502
5112023-12-06T17:41:22.102ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:524
5122023-12-06T17:41:23.007ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:474
5132023-12-06T17:41:23.007ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:769 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:64335
5142023-12-06T17:41:23.016ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 9000 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:64335 req_id = 6fbbc6c5-9bbe-48a4-a252-78dacce9f2f7 response_code = 200 uri = /timesync
5152023-12-06T17:41:23.019ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 2139029761, ip_addr: ::, stratum: 10, ref_time: 1701884477.750389, correction: 0.0 } file = sled-agent/src/rack_setup/service.rs:502
5162023-12-06T17:41:23.019ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:524
5172023-12-06T17:41:23.811ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:474
5182023-12-06T17:41:23.811ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:769 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:53225
5192023-12-06T17:41:23.820ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 8960 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:53225 req_id = c2f0aa29-cda9-4aa9-9573-17f37977dc67 response_code = 200 uri = /timesync
5202023-12-06T17:41:23.824ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 2139029761, ip_addr: ::, stratum: 10, ref_time: 1701884477.750389, correction: 0.0 } file = sled-agent/src/rack_setup/service.rs:502
5212023-12-06T17:41:23.824ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:524
5222023-12-06T17:41:23.923ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::60fc:8ff:fe95:bcc8": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
5232023-12-06T17:41:24.568ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:474
5242023-12-06T17:41:24.568ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:769 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:61285
5252023-12-06T17:41:24.588ZWARNSledAgent (ServiceManager): Updating [wu]tmpx databases failed: Command [/usr/platform/oxide/bin/tmpx -Z] executed and failed with status: exit status: 1 stdout: [zone:global:0] Boot time: 1701884039 Root path: Updating database '/var/adm/wtmpx' First entry looks correct; nothing to do Updating database '/var/adm/utmpx' First entry looks correct; nothing to do [zone:sidecar_softnpu:1] Boot time: 1701884433 Root path: /sidecar/sidecar_softnpu/root Updating database '/sidecar/sidecar_softnpu/root/var/adm/wtmpx' First entry looks correct; nothing to do Updating database '/sidecar/sidecar_softnpu/root/var/adm/utmpx' First entry looks correct; nothing to do [zone:oxz_switch:2] Boot time: 1701884444 Root path: /zone/oxz_switch/root Updating database '/zone/oxz_switch/root/var/adm/wtmpx' First entry looks correct; nothing to do Updating database '/zone/oxz_switch/root/var/adm/utmpx' First entry looks correct; nothing to do [zone:oxz_internal_dns_00f82355-ab6a-44f8-adeb-55a0298e7676:3] Boot time: 1701884454 Root path: /pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_internal_dns_00f82355-ab6a-44f8-adeb-55a0298e7676/root Updating database '/pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_internal_dns_00f82355-ab6a-44f8-adeb-55a0298e7676/root/var/adm/wtmpx' stderr: tmpx: Invalid database: '/pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_internal_dns_00f82355-ab6a-44f8-adeb-55a0298e7676/root/var/adm/wtmpx' (see utmpxname(3C)) file = sled-agent/src/services.rs:2949
5262023-12-06T17:41:24.588ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 19938 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:61285 req_id = 0b043fbe-d150-4c7b-9f23-d9f2049bd962 response_code = 200 uri = /timesync
5272023-12-06T17:41:24.592ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: true, ref_id: 89016161, ip_addr: 5.78.71.97, stratum: 3, ref_time: 1701884483.297359, correction: 0.038588446 } file = sled-agent/src/rack_setup/service.rs:502
5282023-12-06T17:41:24.592ZINFOSledAgent (RSS): Finished setting up Internal DNS and NTP file = sled-agent/src/rack_setup/service.rs:1042
5292023-12-06T17:41:24.623ZINFOSledAgent (RSS): attempting to set up sled's Omicron zones: OmicronZonesConfig { generation: Generation(4), zones: [OmicronZoneConfig { id: 00f82355-ab6a-44f8-adeb-55a0298e7676, underlay_address: fd00:1122:3344:1::1, zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 14b4dc87-ab46-49fb-a4b4-d361ae214c03, kind: External } }, http_address: [fd00:1122:3344:1::1]:5353, dns_address: [fd00:1122:3344:1::1]:53, gz_address: fd00:1122:3344:1::2, gz_address_index: 0 } }, OmicronZoneConfig { id: be8d8c76-3f0a-4bd9-8466-47f0b72402ea, underlay_address: fd00:1122:3344:2::1, zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 24b4dc87-ab46-49fb-a4b4-d361ae214c03, kind: External } }, http_address: [fd00:1122:3344:2::1]:5353, dns_address: [fd00:1122:3344:2::1]:53, gz_address: fd00:1122:3344:2::2, gz_address_index: 1 } }, OmicronZoneConfig { id: ac3d1180-e1c3-477d-a802-3f8bd92e2528, underlay_address: fd00:1122:3344:3::1, zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 31bd71cd-4736-4a12-a387-9b74b050396f, kind: External } }, http_address: [fd00:1122:3344:3::1]:5353, dns_address: [fd00:1122:3344:3::1]:53, gz_address: fd00:1122:3344:3::2, gz_address_index: 2 } }, OmicronZoneConfig { id: 0e075832-9eaa-40c5-a328-dcde3812ee47, underlay_address: fd00:1122:3344:101::1b, zone_type: BoundaryNtp { address: [fd00:1122:3344:101::1b]:123, ntp_servers: ["0.pool.ntp.org"], dns_servers: [1.1.1.1, 9.9.9.9], domain: None, nic: NetworkInterface { id: 1277989f-b18b-4c6b-8166-0b015ebc20df, kind: Service { id: 0e075832-9eaa-40c5-a328-dcde3812ee47 }, name: Name("ntp-0e075832-9eaa-40c5-a328-dcde3812ee47"), ip: 172.30.3.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 143, 64])), subnet: V4(Ipv4Net(Ipv4Network { addr: 172.30.3.0, prefix: 24 })), vni: Vni(100), primary: true, slot: 0 }, snat_cfg: SourceNatConfig { ip: 10.151.2.169, first_port: 0, last_port: 16383 } } }, OmicronZoneConfig { id: ccbe62b8-1461-4dec-a1a5-93b3a022681d, underlay_address: fd00:1122:3344:101::3, zone_type: CockroachDb { address: [fd00:1122:3344:101::3]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 14b4dc87-ab46-49fb-a4b4-d361ae214c03, kind: External } } } }, OmicronZoneConfig { id: c9428da0-985f-4421-8de9-d60f987ba8ab, underlay_address: fd00:1122:3344:101::4, zone_type: CockroachDb { address: [fd00:1122:3344:101::4]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 24b4dc87-ab46-49fb-a4b4-d361ae214c03, kind: External } } } }, OmicronZoneConfig { id: 8f3a1d19-d2a3-45ba-b935-68f0390bcdf8, underlay_address: fd00:1122:3344:101::5, zone_type: CockroachDb { address: [fd00:1122:3344:101::5]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 31bd71cd-4736-4a12-a387-9b74b050396f, kind: External } } } }, OmicronZoneConfig { id: 5c732813-f138-434d-bcc4-6ed72087dd45, underlay_address: fd00:1122:3344:101::6, zone_type: CockroachDb { address: [fd00:1122:3344:101::6]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 616b26df-e62a-4c68-b506-f4a923d8aaf7, kind: External } } } }, OmicronZoneConfig { id: 8caca53a-45ee-4bbf-9f3b-bf2a9c9243ef, underlay_address: fd00:1122:3344:101::7, zone_type: CockroachDb { address: [fd00:1122:3344:101::7]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: cd70d7f6-2354-4bf2-8012-55bf9eaf7930, kind: External } } } }] } file = sled-agent/src/rack_setup/service.rs:299 sled_address = [fd00:1122:3344:101::1]:12345
5302023-12-06T17:41:24.623ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:769 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:53633
5312023-12-06T17:41:24.623ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 00f82355-ab6a-44f8-adeb-55a0298e7676, dataset_name: DatasetName { pool_name: ZpoolName { id: 14b4dc87-ab46-49fb-a4b4-d361ae214c03, kind: External }, kind: InternalDns }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } }) file = sled-storage/src/manager.rs:346
5322023-12-06T17:41:24.623ZINFOSledAgent (StorageManager): add_dataset: NewFilesystemRequest { dataset_id: 00f82355-ab6a-44f8-adeb-55a0298e7676, dataset_name: DatasetName { pool_name: ZpoolName { id: 14b4dc87-ab46-49fb-a4b4-d361ae214c03, kind: External }, kind: InternalDns }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } } file = sled-storage/src/manager.rs:564
5332023-12-06T17:41:24.643ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: be8d8c76-3f0a-4bd9-8466-47f0b72402ea, dataset_name: DatasetName { pool_name: ZpoolName { id: 24b4dc87-ab46-49fb-a4b4-d361ae214c03, kind: External }, kind: InternalDns }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } }) file = sled-storage/src/manager.rs:346
5342023-12-06T17:41:24.644ZINFOSledAgent (StorageManager): add_dataset: NewFilesystemRequest { dataset_id: be8d8c76-3f0a-4bd9-8466-47f0b72402ea, dataset_name: DatasetName { pool_name: ZpoolName { id: 24b4dc87-ab46-49fb-a4b4-d361ae214c03, kind: External }, kind: InternalDns }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } } file = sled-storage/src/manager.rs:564
5352023-12-06T17:41:24.662ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: ac3d1180-e1c3-477d-a802-3f8bd92e2528, dataset_name: DatasetName { pool_name: ZpoolName { id: 31bd71cd-4736-4a12-a387-9b74b050396f, kind: External }, kind: InternalDns }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } }) file = sled-storage/src/manager.rs:346
5362023-12-06T17:41:24.662ZINFOSledAgent (StorageManager): add_dataset: NewFilesystemRequest { dataset_id: ac3d1180-e1c3-477d-a802-3f8bd92e2528, dataset_name: DatasetName { pool_name: ZpoolName { id: 31bd71cd-4736-4a12-a387-9b74b050396f, kind: External }, kind: InternalDns }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } } file = sled-storage/src/manager.rs:564
5372023-12-06T17:41:24.680ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: ccbe62b8-1461-4dec-a1a5-93b3a022681d, dataset_name: DatasetName { pool_name: ZpoolName { id: 14b4dc87-ab46-49fb-a4b4-d361ae214c03, kind: External }, kind: CockroachDb }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } }) file = sled-storage/src/manager.rs:346
5382023-12-06T17:41:24.680ZINFOSledAgent (StorageManager): add_dataset: NewFilesystemRequest { dataset_id: ccbe62b8-1461-4dec-a1a5-93b3a022681d, dataset_name: DatasetName { pool_name: ZpoolName { id: 14b4dc87-ab46-49fb-a4b4-d361ae214c03, kind: External }, kind: CockroachDb }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } } file = sled-storage/src/manager.rs:564
5392023-12-06T17:41:24.731ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: c9428da0-985f-4421-8de9-d60f987ba8ab, dataset_name: DatasetName { pool_name: ZpoolName { id: 24b4dc87-ab46-49fb-a4b4-d361ae214c03, kind: External }, kind: CockroachDb }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } }) file = sled-storage/src/manager.rs:346
5402023-12-06T17:41:24.731ZINFOSledAgent (StorageManager): add_dataset: NewFilesystemRequest { dataset_id: c9428da0-985f-4421-8de9-d60f987ba8ab, dataset_name: DatasetName { pool_name: ZpoolName { id: 24b4dc87-ab46-49fb-a4b4-d361ae214c03, kind: External }, kind: CockroachDb }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } } file = sled-storage/src/manager.rs:564
5412023-12-06T17:41:24.779ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 8f3a1d19-d2a3-45ba-b935-68f0390bcdf8, dataset_name: DatasetName { pool_name: ZpoolName { id: 31bd71cd-4736-4a12-a387-9b74b050396f, kind: External }, kind: CockroachDb }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } }) file = sled-storage/src/manager.rs:346
5422023-12-06T17:41:24.779ZINFOSledAgent (StorageManager): add_dataset: NewFilesystemRequest { dataset_id: 8f3a1d19-d2a3-45ba-b935-68f0390bcdf8, dataset_name: DatasetName { pool_name: ZpoolName { id: 31bd71cd-4736-4a12-a387-9b74b050396f, kind: External }, kind: CockroachDb }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } } file = sled-storage/src/manager.rs:564
5432023-12-06T17:41:24.826ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 5c732813-f138-434d-bcc4-6ed72087dd45, dataset_name: DatasetName { pool_name: ZpoolName { id: 616b26df-e62a-4c68-b506-f4a923d8aaf7, kind: External }, kind: CockroachDb }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } }) file = sled-storage/src/manager.rs:346
5442023-12-06T17:41:24.826ZINFOSledAgent (StorageManager): add_dataset: NewFilesystemRequest { dataset_id: 5c732813-f138-434d-bcc4-6ed72087dd45, dataset_name: DatasetName { pool_name: ZpoolName { id: 616b26df-e62a-4c68-b506-f4a923d8aaf7, kind: External }, kind: CockroachDb }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } } file = sled-storage/src/manager.rs:564
5452023-12-06T17:41:24.875ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 8caca53a-45ee-4bbf-9f3b-bf2a9c9243ef, dataset_name: DatasetName { pool_name: ZpoolName { id: cd70d7f6-2354-4bf2-8012-55bf9eaf7930, kind: External }, kind: CockroachDb }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } }) file = sled-storage/src/manager.rs:346
5462023-12-06T17:41:24.875ZINFOSledAgent (StorageManager): add_dataset: NewFilesystemRequest { dataset_id: 8caca53a-45ee-4bbf-9f3b-bf2a9c9243ef, dataset_name: DatasetName { pool_name: ZpoolName { id: cd70d7f6-2354-4bf2-8012-55bf9eaf7930, kind: External }, kind: CockroachDb }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } } file = sled-storage/src/manager.rs:564
5472023-12-06T17:41:24.928ZINFOSledAgent (StorageManager): Received GetLatestResources(Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) }) file = sled-storage/src/manager.rs:346
5482023-12-06T17:41:24.928ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/a462a7f7-b628-40fe-80ff-4e4189e2d62b/config/omicron-zones.json file = common/src/ledger.rs:176
5492023-12-06T17:41:24.928ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/b462a7f7-b628-40fe-80ff-4e4189e2d62b/config/omicron-zones.json file = common/src/ledger.rs:176
5502023-12-06T17:41:24.928ZINFOSledAgent (StorageManager): Received GetLatestResources(Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) }) file = sled-storage/src/manager.rs:346
5512023-12-06T17:41:24.936ZINFOSledAgent (StorageManager): Received GetLatestResources(Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) }) file = sled-storage/src/manager.rs:346
5522023-12-06T17:41:24.936ZINFOSledAgent (StorageManager): Received GetLatestResources(Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) }) file = sled-storage/src/manager.rs:346
5532023-12-06T17:41:24.936ZINFOSledAgent (StorageManager): Received GetLatestResources(Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) }) file = sled-storage/src/manager.rs:346
5542023-12-06T17:41:24.936ZINFOSledAgent (StorageManager): Received GetLatestResources(Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) }) file = sled-storage/src/manager.rs:346
5552023-12-06T17:41:24.936ZINFOSledAgent (StorageManager): Received GetLatestResources(Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) }) file = sled-storage/src/manager.rs:346
5562023-12-06T17:41:25.027ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_cockroachdb_ccbe62b8-1461-4dec-a1a5-93b3a022681d file = illumos-utils/src/zone.rs:314
5572023-12-06T17:41:25.031ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_cockroachdb_8caca53a-45ee-4bbf-9f3b-bf2a9c9243ef file = illumos-utils/src/zone.rs:314
5582023-12-06T17:41:25.036ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_cockroachdb_8f3a1d19-d2a3-45ba-b935-68f0390bcdf8 file = illumos-utils/src/zone.rs:314
5592023-12-06T17:41:25.040ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_cockroachdb_c9428da0-985f-4421-8de9-d60f987ba8ab file = illumos-utils/src/zone.rs:314
5602023-12-06T17:41:25.045ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_cockroachdb_5c732813-f138-434d-bcc4-6ed72087dd45 file = illumos-utils/src/zone.rs:314
5612023-12-06T17:41:25.049ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_cockroachdb_ccbe62b8-1461-4dec-a1a5-93b3a022681d file = illumos-utils/src/zone.rs:353
5622023-12-06T17:41:25.054ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_cockroachdb_8caca53a-45ee-4bbf-9f3b-bf2a9c9243ef file = illumos-utils/src/zone.rs:353
5632023-12-06T17:41:25.058ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_cockroachdb_8f3a1d19-d2a3-45ba-b935-68f0390bcdf8 file = illumos-utils/src/zone.rs:353
5642023-12-06T17:41:25.063ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_cockroachdb_c9428da0-985f-4421-8de9-d60f987ba8ab file = illumos-utils/src/zone.rs:353
5652023-12-06T17:41:25.067ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_cockroachdb_5c732813-f138-434d-bcc4-6ed72087dd45 file = illumos-utils/src/zone.rs:353
5662023-12-06T17:41:28.551ZINFOSledAgent (ServiceManager): Profile for oxz_cockroachdb_8caca53a-45ee-4bbf-9f3b-bf2a9c9243ef: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/cockroachdb"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value="oxControlService6"/> <propval type="astring" name="gateway" value="fd00:1122:3344:101::1"/> <propval type="astring" name="listen_addr" value="fd00:1122:3344:101::7"/> <propval type="astring" name="listen_port" value="32221"/> <propval type="astring" name="store" value="/data"/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/install"> <property_group type="application" name="install_props"> <property type="net_address" name="nameserver"> <net_address_list> <value_node value="fd00:1122:3344:1::1" /> <value_node value="fd00:1122:3344:3::1" /> <value_node value="fd00:1122:3344:2::1" /> </net_address_list> </property> </property_group> <instance enabled="true" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
5672023-12-06T17:41:28.551ZINFOSledAgent (ServiceManager): Zone booting file = illumos-utils/src/running_zone.rs:512 zone = oxz_cockroachdb_8caca53a-45ee-4bbf-9f3b-bf2a9c9243ef
5682023-12-06T17:41:28.714ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_8caca53a-45ee-4bbf-9f3b-bf2a9c9243ef") failed: Property not found. retry in 45.926322ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_8caca53a-45ee-4bbf-9f3b-bf2a9c9243ef
5692023-12-06T17:41:28.717ZINFOSledAgent (ServiceManager): Profile for oxz_cockroachdb_c9428da0-985f-4421-8de9-d60f987ba8ab: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/cockroachdb"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value="oxControlService8"/> <propval type="astring" name="gateway" value="fd00:1122:3344:101::1"/> <propval type="astring" name="listen_addr" value="fd00:1122:3344:101::4"/> <propval type="astring" name="listen_port" value="32221"/> <propval type="astring" name="store" value="/data"/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/install"> <property_group type="application" name="install_props"> <property type="net_address" name="nameserver"> <net_address_list> <value_node value="fd00:1122:3344:1::1" /> <value_node value="fd00:1122:3344:3::1" /> <value_node value="fd00:1122:3344:2::1" /> </net_address_list> </property> </property_group> <instance enabled="true" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
5702023-12-06T17:41:28.717ZINFOSledAgent (ServiceManager): Zone booting file = illumos-utils/src/running_zone.rs:512 zone = oxz_cockroachdb_c9428da0-985f-4421-8de9-d60f987ba8ab
5712023-12-06T17:41:28.752ZINFOSledAgent (ServiceManager): Profile for oxz_cockroachdb_5c732813-f138-434d-bcc4-6ed72087dd45: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/cockroachdb"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value="oxControlService9"/> <propval type="astring" name="gateway" value="fd00:1122:3344:101::1"/> <propval type="astring" name="listen_addr" value="fd00:1122:3344:101::6"/> <propval type="astring" name="listen_port" value="32221"/> <propval type="astring" name="store" value="/data"/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/install"> <property_group type="application" name="install_props"> <property type="net_address" name="nameserver"> <net_address_list> <value_node value="fd00:1122:3344:1::1" /> <value_node value="fd00:1122:3344:3::1" /> <value_node value="fd00:1122:3344:2::1" /> </net_address_list> </property> </property_group> <instance enabled="true" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
5722023-12-06T17:41:28.752ZINFOSledAgent (ServiceManager): Zone booting file = illumos-utils/src/running_zone.rs:512 zone = oxz_cockroachdb_5c732813-f138-434d-bcc4-6ed72087dd45
5732023-12-06T17:41:28.769ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_8caca53a-45ee-4bbf-9f3b-bf2a9c9243ef") failed: Property not found. retry in 123.60862ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_8caca53a-45ee-4bbf-9f3b-bf2a9c9243ef
5742023-12-06T17:41:28.923ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_8caca53a-45ee-4bbf-9f3b-bf2a9c9243ef") failed: Property not found. retry in 163.641294ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_8caca53a-45ee-4bbf-9f3b-bf2a9c9243ef
5752023-12-06T17:41:28.935ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::60fc:8ff:fe95:bcc8": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
5762023-12-06T17:41:28.974ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_c9428da0-985f-4421-8de9-d60f987ba8ab") failed: Property not found. retry in 39.590133ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_c9428da0-985f-4421-8de9-d60f987ba8ab
5772023-12-06T17:41:28.992ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_5c732813-f138-434d-bcc4-6ed72087dd45") failed: Property not found. retry in 51.494887ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_5c732813-f138-434d-bcc4-6ed72087dd45
5782023-12-06T17:41:29.035ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_c9428da0-985f-4421-8de9-d60f987ba8ab") failed: Property not found. retry in 125.498182ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_c9428da0-985f-4421-8de9-d60f987ba8ab
5792023-12-06T17:41:29.093ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_5c732813-f138-434d-bcc4-6ed72087dd45") failed: Property not found. retry in 129.862173ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_5c732813-f138-434d-bcc4-6ed72087dd45
5802023-12-06T17:41:29.185ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_8caca53a-45ee-4bbf-9f3b-bf2a9c9243ef") failed: Property not found. retry in 524.024665ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_8caca53a-45ee-4bbf-9f3b-bf2a9c9243ef
5812023-12-06T17:41:29.268ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_c9428da0-985f-4421-8de9-d60f987ba8ab") failed: Property not found. retry in 126.884947ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_c9428da0-985f-4421-8de9-d60f987ba8ab
5822023-12-06T17:41:29.390ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_5c732813-f138-434d-bcc4-6ed72087dd45") failed: Property not found. retry in 246.074289ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_5c732813-f138-434d-bcc4-6ed72087dd45
5832023-12-06T17:41:29.456ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_c9428da0-985f-4421-8de9-d60f987ba8ab") failed: Property not found. retry in 557.957861ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_c9428da0-985f-4421-8de9-d60f987ba8ab
5842023-12-06T17:41:29.649ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_5c732813-f138-434d-bcc4-6ed72087dd45") failed: Property not found. retry in 426.703176ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_5c732813-f138-434d-bcc4-6ed72087dd45
5852023-12-06T17:41:29.730ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_8caca53a-45ee-4bbf-9f3b-bf2a9c9243ef") failed: Property not found. retry in 1.168007367s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_8caca53a-45ee-4bbf-9f3b-bf2a9c9243ef
5862023-12-06T17:41:30.038ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_c9428da0-985f-4421-8de9-d60f987ba8ab") failed: Property not found. retry in 931.202213ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_c9428da0-985f-4421-8de9-d60f987ba8ab
5872023-12-06T17:41:30.115ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_5c732813-f138-434d-bcc4-6ed72087dd45") failed: Property not found. retry in 841.30258ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_5c732813-f138-434d-bcc4-6ed72087dd45
5882023-12-06T17:41:30.381ZINFOSledAgent (ServiceManager): Profile for oxz_cockroachdb_ccbe62b8-1461-4dec-a1a5-93b3a022681d: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/cockroachdb"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value="oxControlService5"/> <propval type="astring" name="gateway" value="fd00:1122:3344:101::1"/> <propval type="astring" name="listen_addr" value="fd00:1122:3344:101::3"/> <propval type="astring" name="listen_port" value="32221"/> <propval type="astring" name="store" value="/data"/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/install"> <property_group type="application" name="install_props"> <property type="net_address" name="nameserver"> <net_address_list> <value_node value="fd00:1122:3344:1::1" /> <value_node value="fd00:1122:3344:3::1" /> <value_node value="fd00:1122:3344:2::1" /> </net_address_list> </property> </property_group> <instance enabled="true" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
5892023-12-06T17:41:30.381ZINFOSledAgent (ServiceManager): Zone booting file = illumos-utils/src/running_zone.rs:512 zone = oxz_cockroachdb_ccbe62b8-1461-4dec-a1a5-93b3a022681d
5902023-12-06T17:41:30.533ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_ccbe62b8-1461-4dec-a1a5-93b3a022681d") failed: Property not found. retry in 68.079936ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_ccbe62b8-1461-4dec-a1a5-93b3a022681d
5912023-12-06T17:41:30.613ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_ccbe62b8-1461-4dec-a1a5-93b3a022681d") failed: Property not found. retry in 134.467066ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_ccbe62b8-1461-4dec-a1a5-93b3a022681d
5922023-12-06T17:41:30.680ZINFOSledAgent (ServiceManager): Profile for oxz_cockroachdb_8f3a1d19-d2a3-45ba-b935-68f0390bcdf8: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/cockroachdb"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value="oxControlService7"/> <propval type="astring" name="gateway" value="fd00:1122:3344:101::1"/> <propval type="astring" name="listen_addr" value="fd00:1122:3344:101::5"/> <propval type="astring" name="listen_port" value="32221"/> <propval type="astring" name="store" value="/data"/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/install"> <property_group type="application" name="install_props"> <property type="net_address" name="nameserver"> <net_address_list> <value_node value="fd00:1122:3344:1::1" /> <value_node value="fd00:1122:3344:3::1" /> <value_node value="fd00:1122:3344:2::1" /> </net_address_list> </property> </property_group> <instance enabled="true" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
5932023-12-06T17:41:30.680ZINFOSledAgent (ServiceManager): Zone booting file = illumos-utils/src/running_zone.rs:512 zone = oxz_cockroachdb_8f3a1d19-d2a3-45ba-b935-68f0390bcdf8
5942023-12-06T17:41:30.762ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_ccbe62b8-1461-4dec-a1a5-93b3a022681d") failed: Property not found. retry in 103.982195ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_ccbe62b8-1461-4dec-a1a5-93b3a022681d
5952023-12-06T17:41:30.871ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_8f3a1d19-d2a3-45ba-b935-68f0390bcdf8") failed: Property not found. retry in 30.080616ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_8f3a1d19-d2a3-45ba-b935-68f0390bcdf8
5962023-12-06T17:41:30.879ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_ccbe62b8-1461-4dec-a1a5-93b3a022681d") failed: Property not found. retry in 433.449655ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_ccbe62b8-1461-4dec-a1a5-93b3a022681d
5972023-12-06T17:41:30.909ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_8caca53a-45ee-4bbf-9f3b-bf2a9c9243ef") failed: Property not found. retry in 1.074891763s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_8caca53a-45ee-4bbf-9f3b-bf2a9c9243ef
5982023-12-06T17:41:30.917ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_8f3a1d19-d2a3-45ba-b935-68f0390bcdf8") failed: Property not found. retry in 70.278865ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_8f3a1d19-d2a3-45ba-b935-68f0390bcdf8
5992023-12-06T17:41:30.970ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_5c732813-f138-434d-bcc4-6ed72087dd45") failed: Property not found. retry in 537.428335ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_5c732813-f138-434d-bcc4-6ed72087dd45
6002023-12-06T17:41:30.982ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_c9428da0-985f-4421-8de9-d60f987ba8ab") failed: Property not found. retry in 640.326267ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_c9428da0-985f-4421-8de9-d60f987ba8ab
6012023-12-06T17:41:30.996ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_8f3a1d19-d2a3-45ba-b935-68f0390bcdf8") failed: Property not found. retry in 112.641774ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_8f3a1d19-d2a3-45ba-b935-68f0390bcdf8
6022023-12-06T17:41:31.120ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_8f3a1d19-d2a3-45ba-b935-68f0390bcdf8") failed: Property not found. retry in 596.037721ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_8f3a1d19-d2a3-45ba-b935-68f0390bcdf8
6032023-12-06T17:41:31.323ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_ccbe62b8-1461-4dec-a1a5-93b3a022681d") failed: Property not found. retry in 597.094416ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_ccbe62b8-1461-4dec-a1a5-93b3a022681d
6042023-12-06T17:41:31.519ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_5c732813-f138-434d-bcc4-6ed72087dd45") failed: Property not found. retry in 1.340754607s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_5c732813-f138-434d-bcc4-6ed72087dd45
6052023-12-06T17:41:31.633ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_c9428da0-985f-4421-8de9-d60f987ba8ab") failed: Property not found. retry in 990.618619ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_c9428da0-985f-4421-8de9-d60f987ba8ab
6062023-12-06T17:41:31.728ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_8f3a1d19-d2a3-45ba-b935-68f0390bcdf8") failed: Property not found. retry in 839.754979ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_8f3a1d19-d2a3-45ba-b935-68f0390bcdf8
6072023-12-06T17:41:31.933ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_ccbe62b8-1461-4dec-a1a5-93b3a022681d") failed: Property not found. retry in 1.006496852s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_ccbe62b8-1461-4dec-a1a5-93b3a022681d
6082023-12-06T17:41:31.995ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_8caca53a-45ee-4bbf-9f3b-bf2a9c9243ef") failed: Property not found. retry in 1.006902139s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_8caca53a-45ee-4bbf-9f3b-bf2a9c9243ef
6092023-12-06T17:41:32.578ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_8f3a1d19-d2a3-45ba-b935-68f0390bcdf8") failed: Property not found. retry in 1.152765196s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_8f3a1d19-d2a3-45ba-b935-68f0390bcdf8
6102023-12-06T17:41:32.635ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_c9428da0-985f-4421-8de9-d60f987ba8ab") failed: Property not found. retry in 783.904569ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_c9428da0-985f-4421-8de9-d60f987ba8ab
6112023-12-06T17:41:32.872ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_5c732813-f138-434d-bcc4-6ed72087dd45") failed: Property not found. retry in 1.043413497s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_5c732813-f138-434d-bcc4-6ed72087dd45
6122023-12-06T17:41:32.949ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_ccbe62b8-1461-4dec-a1a5-93b3a022681d") failed: Property not found. retry in 1.29876064s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_ccbe62b8-1461-4dec-a1a5-93b3a022681d
6132023-12-06T17:41:33.014ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_8caca53a-45ee-4bbf-9f3b-bf2a9c9243ef") failed: Property not found. retry in 815.16897ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_8caca53a-45ee-4bbf-9f3b-bf2a9c9243ef
6142023-12-06T17:41:33.430ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_c9428da0-985f-4421-8de9-d60f987ba8ab") failed: Property not found. retry in 625.224954ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_c9428da0-985f-4421-8de9-d60f987ba8ab
6152023-12-06T17:41:33.743ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_8f3a1d19-d2a3-45ba-b935-68f0390bcdf8") failed: Property not found. retry in 1.461792641s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_8f3a1d19-d2a3-45ba-b935-68f0390bcdf8
6162023-12-06T17:41:33.841ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_8caca53a-45ee-4bbf-9f3b-bf2a9c9243ef") failed: Property not found. retry in 1.322929271s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_8caca53a-45ee-4bbf-9f3b-bf2a9c9243ef
6172023-12-06T17:41:33.926ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_5c732813-f138-434d-bcc4-6ed72087dd45") failed: Property not found. retry in 709.808447ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_5c732813-f138-434d-bcc4-6ed72087dd45
6182023-12-06T17:41:33.950ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::60fc:8ff:fe95:bcc8": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
6192023-12-06T17:41:34.067ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_c9428da0-985f-4421-8de9-d60f987ba8ab") failed: Property not found. retry in 504.279475ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_c9428da0-985f-4421-8de9-d60f987ba8ab
6202023-12-06T17:41:34.260ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_ccbe62b8-1461-4dec-a1a5-93b3a022681d") failed: Property not found. retry in 964.502566ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_ccbe62b8-1461-4dec-a1a5-93b3a022681d
6212023-12-06T17:41:34.583ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_c9428da0-985f-4421-8de9-d60f987ba8ab") failed: Property not found. retry in 889.50833ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_c9428da0-985f-4421-8de9-d60f987ba8ab
6222023-12-06T17:41:34.648ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_5c732813-f138-434d-bcc4-6ed72087dd45") failed: Property not found. retry in 1.430796319s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_5c732813-f138-434d-bcc4-6ed72087dd45
6232023-12-06T17:41:35.177ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_8caca53a-45ee-4bbf-9f3b-bf2a9c9243ef") failed: Property not found. retry in 1.470815828s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_8caca53a-45ee-4bbf-9f3b-bf2a9c9243ef
6242023-12-06T17:41:35.218ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_8f3a1d19-d2a3-45ba-b935-68f0390bcdf8") failed: Property not found. retry in 717.929855ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_8f3a1d19-d2a3-45ba-b935-68f0390bcdf8
6252023-12-06T17:41:35.237ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_ccbe62b8-1461-4dec-a1a5-93b3a022681d") failed: Property not found. retry in 690.97221ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_ccbe62b8-1461-4dec-a1a5-93b3a022681d
6262023-12-06T17:41:35.485ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_c9428da0-985f-4421-8de9-d60f987ba8ab") failed: Property not found. retry in 790.449884ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_c9428da0-985f-4421-8de9-d60f987ba8ab
6272023-12-06T17:41:35.939ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_ccbe62b8-1461-4dec-a1a5-93b3a022681d") failed: Property not found. retry in 701.637834ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_ccbe62b8-1461-4dec-a1a5-93b3a022681d
6282023-12-06T17:41:35.947ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_8f3a1d19-d2a3-45ba-b935-68f0390bcdf8") failed: Property not found. retry in 776.213237ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_8f3a1d19-d2a3-45ba-b935-68f0390bcdf8
6292023-12-06T17:41:36.093ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_5c732813-f138-434d-bcc4-6ed72087dd45") failed: Property not found. retry in 818.257328ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_5c732813-f138-434d-bcc4-6ed72087dd45
6302023-12-06T17:41:36.181ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:769 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:62471
6312023-12-06T17:41:36.181ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 68 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:62471 req_id = 27897e74-c4e8-4b86-bd61-798af7b66fb6 response_code = 200 uri = /baseboard
6322023-12-06T17:41:36.287ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_c9428da0-985f-4421-8de9-d60f987ba8ab") failed: Property not found. retry in 648.186148ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_c9428da0-985f-4421-8de9-d60f987ba8ab
6332023-12-06T17:41:36.652ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_ccbe62b8-1461-4dec-a1a5-93b3a022681d") failed: Property not found. retry in 1.16288358s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_ccbe62b8-1461-4dec-a1a5-93b3a022681d
6342023-12-06T17:41:36.661ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_8caca53a-45ee-4bbf-9f3b-bf2a9c9243ef") failed: Property not found. retry in 1.3145561s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_8caca53a-45ee-4bbf-9f3b-bf2a9c9243ef
6352023-12-06T17:41:36.734ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_8f3a1d19-d2a3-45ba-b935-68f0390bcdf8") failed: Property not found. retry in 1.386279833s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_8f3a1d19-d2a3-45ba-b935-68f0390bcdf8
6362023-12-06T17:41:36.923ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_5c732813-f138-434d-bcc4-6ed72087dd45") failed: Property not found. retry in 737.115995ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_5c732813-f138-434d-bcc4-6ed72087dd45
6372023-12-06T17:41:36.948ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_c9428da0-985f-4421-8de9-d60f987ba8ab") failed: Property not found. retry in 1.006811482s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_c9428da0-985f-4421-8de9-d60f987ba8ab
6382023-12-06T17:41:37.672ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_5c732813-f138-434d-bcc4-6ed72087dd45") failed: Property not found. retry in 1.040538456s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_5c732813-f138-434d-bcc4-6ed72087dd45
6392023-12-06T17:41:37.827ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_ccbe62b8-1461-4dec-a1a5-93b3a022681d") failed: Property not found. retry in 787.704339ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_ccbe62b8-1461-4dec-a1a5-93b3a022681d
6402023-12-06T17:41:37.967ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_c9428da0-985f-4421-8de9-d60f987ba8ab") failed: Property not found. retry in 1.489360619s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_c9428da0-985f-4421-8de9-d60f987ba8ab
6412023-12-06T17:41:37.988ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_8caca53a-45ee-4bbf-9f3b-bf2a9c9243ef") failed: Property not found. retry in 894.247591ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_8caca53a-45ee-4bbf-9f3b-bf2a9c9243ef
6422023-12-06T17:41:38.133ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_8f3a1d19-d2a3-45ba-b935-68f0390bcdf8") failed: Property not found. retry in 799.767457ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_8f3a1d19-d2a3-45ba-b935-68f0390bcdf8
6432023-12-06T17:41:38.627ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_ccbe62b8-1461-4dec-a1a5-93b3a022681d") failed: Property not found. retry in 1.282101887s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_ccbe62b8-1461-4dec-a1a5-93b3a022681d
6442023-12-06T17:41:38.725ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_5c732813-f138-434d-bcc4-6ed72087dd45") failed: Property not found. retry in 1.072343351s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_5c732813-f138-434d-bcc4-6ed72087dd45
6452023-12-06T17:41:38.894ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_8caca53a-45ee-4bbf-9f3b-bf2a9c9243ef") failed: Property not found. retry in 576.228662ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_8caca53a-45ee-4bbf-9f3b-bf2a9c9243ef
6462023-12-06T17:41:38.942ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_8f3a1d19-d2a3-45ba-b935-68f0390bcdf8") failed: Property not found. retry in 797.102006ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_8f3a1d19-d2a3-45ba-b935-68f0390bcdf8
6472023-12-06T17:41:38.966ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::60fc:8ff:fe95:bcc8": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
6482023-12-06T17:41:39.472ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_c9428da0-985f-4421-8de9-d60f987ba8ab") failed: Property not found. retry in 1.366388425s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_c9428da0-985f-4421-8de9-d60f987ba8ab
6492023-12-06T17:41:39.481ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_8caca53a-45ee-4bbf-9f3b-bf2a9c9243ef") failed: Property not found. retry in 1.020252807s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_8caca53a-45ee-4bbf-9f3b-bf2a9c9243ef
6502023-12-06T17:41:39.752ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_8f3a1d19-d2a3-45ba-b935-68f0390bcdf8") failed: Property not found. retry in 820.086151ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_8f3a1d19-d2a3-45ba-b935-68f0390bcdf8
6512023-12-06T17:41:40.866ZINFOSledAgent (ServiceManager): Writing ledger to /pool/int/a462a7f7-b628-40fe-80ff-4e4189e2d62b/config/.omicron-zones.json.tmp file = common/src/ledger.rs:198
6522023-12-06T17:41:40.867ZINFOSledAgent (ServiceManager): Writing ledger to /pool/int/b462a7f7-b628-40fe-80ff-4e4189e2d62b/config/.omicron-zones.json.tmp file = common/src/ledger.rs:198
6532023-12-06T17:41:40.867ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 16203694 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::1]:53633 req_id = 7bc36e09-b028-475b-b0ca-0e611b5b64c8 response_code = 204 uri = /omicron-zones
6542023-12-06T17:41:40.900ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:769 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:53898
6552023-12-06T17:41:40.928ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = Failed to get address: Failed to get address for name oxControlService9/omicron6 in oxz_cockroachdb_5c732813-f138-434d-bcc4-6ed72087dd45: Zone execution error: Command [/usr/sbin/zlogin oxz_cockroachdb_5c732813-f138-434d-bcc4-6ed72087dd45 /usr/sbin/ipadm show-addr -p -o ADDR oxControlService9/omicron6] executed and failed with status: exit status: 1 stdout: \n stderr: ipadm: Address object not found file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:841 latency_us = 28081 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::1]:53898 req_id = 4e22f3d5-99d5-4382-8b79-f8fde26a2745 response_code = 500 uri = /cockroachdb
6562023-12-06T17:41:40.928ZWARNSledAgent (RSS): Failed to initialize CockroachDB error = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "4e22f3d5-99d5-4382-8b79-f8fde26a2745", "content-length": "124", "date": "Wed, 06 Dec 2023 17:41:40 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "4e22f3d5-99d5-4382-8b79-f8fde26a2745" } file = sled-agent/src/rack_setup/service.rs:758 retry_after = 76.623801ms
6572023-12-06T17:41:41.032ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = Failed to get address: Failed to get address for name oxControlService9/omicron6 in oxz_cockroachdb_5c732813-f138-434d-bcc4-6ed72087dd45: Zone execution error: Command [/usr/sbin/zlogin oxz_cockroachdb_5c732813-f138-434d-bcc4-6ed72087dd45 /usr/sbin/ipadm show-addr -p -o ADDR oxControlService9/omicron6] executed and failed with status: exit status: 1 stdout: \n stderr: ipadm: Address object not found file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:841 latency_us = 26782 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::1]:53898 req_id = 75824856-0aa6-4f30-8846-a8ba4b3bed2f response_code = 500 uri = /cockroachdb
6582023-12-06T17:41:41.033ZWARNSledAgent (RSS): Failed to initialize CockroachDB error = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "75824856-0aa6-4f30-8846-a8ba4b3bed2f", "content-length": "124", "date": "Wed, 06 Dec 2023 17:41:40 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "75824856-0aa6-4f30-8846-a8ba4b3bed2f" } file = sled-agent/src/rack_setup/service.rs:758 retry_after = 93.802924ms
6592023-12-06T17:41:41.155ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = Failed to get address: Failed to get address for name oxControlService9/omicron6 in oxz_cockroachdb_5c732813-f138-434d-bcc4-6ed72087dd45: Zone execution error: Command [/usr/sbin/zlogin oxz_cockroachdb_5c732813-f138-434d-bcc4-6ed72087dd45 /usr/sbin/ipadm show-addr -p -o ADDR oxControlService9/omicron6] executed and failed with status: exit status: 1 stdout: \n stderr: ipadm: Address object not found file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:841 latency_us = 27581 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::1]:53898 req_id = 8fc4cbdd-48c1-484b-8b1a-2b44f52f1831 response_code = 500 uri = /cockroachdb
6602023-12-06T17:41:41.156ZWARNSledAgent (RSS): Failed to initialize CockroachDB error = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "8fc4cbdd-48c1-484b-8b1a-2b44f52f1831", "content-length": "124", "date": "Wed, 06 Dec 2023 17:41:40 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "8fc4cbdd-48c1-484b-8b1a-2b44f52f1831" } file = sled-agent/src/rack_setup/service.rs:758 retry_after = 174.321138ms
6612023-12-06T17:41:41.358ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = Failed to get address: Failed to get address for name oxControlService9/omicron6 in oxz_cockroachdb_5c732813-f138-434d-bcc4-6ed72087dd45: Zone execution error: Command [/usr/sbin/zlogin oxz_cockroachdb_5c732813-f138-434d-bcc4-6ed72087dd45 /usr/sbin/ipadm show-addr -p -o ADDR oxControlService9/omicron6] executed and failed with status: exit status: 1 stdout: \n stderr: ipadm: Address object not found file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:841 latency_us = 27300 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::1]:53898 req_id = 7dbaf723-7b1d-4a9b-8271-3e4d81523122 response_code = 500 uri = /cockroachdb
6622023-12-06T17:41:41.358ZWARNSledAgent (RSS): Failed to initialize CockroachDB error = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "7dbaf723-7b1d-4a9b-8271-3e4d81523122", "content-length": "124", "date": "Wed, 06 Dec 2023 17:41:40 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "7dbaf723-7b1d-4a9b-8271-3e4d81523122" } file = sled-agent/src/rack_setup/service.rs:758 retry_after = 88.58445ms
6632023-12-06T17:41:41.474ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = Failed to get address: Failed to get address for name oxControlService9/omicron6 in oxz_cockroachdb_5c732813-f138-434d-bcc4-6ed72087dd45: Zone execution error: Command [/usr/sbin/zlogin oxz_cockroachdb_5c732813-f138-434d-bcc4-6ed72087dd45 /usr/sbin/ipadm show-addr -p -o ADDR oxControlService9/omicron6] executed and failed with status: exit status: 1 stdout: \n stderr: ipadm: Address object not found file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:841 latency_us = 26964 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::1]:53898 req_id = 94d1cbf1-ebc2-4139-9778-341efb5f131f response_code = 500 uri = /cockroachdb
6642023-12-06T17:41:41.475ZWARNSledAgent (RSS): Failed to initialize CockroachDB error = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "94d1cbf1-ebc2-4139-9778-341efb5f131f", "content-length": "124", "date": "Wed, 06 Dec 2023 17:41:40 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "94d1cbf1-ebc2-4139-9778-341efb5f131f" } file = sled-agent/src/rack_setup/service.rs:758 retry_after = 108.082167ms
6652023-12-06T17:41:41.610ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = Failed to get address: Failed to get address for name oxControlService9/omicron6 in oxz_cockroachdb_5c732813-f138-434d-bcc4-6ed72087dd45: Zone execution error: Command [/usr/sbin/zlogin oxz_cockroachdb_5c732813-f138-434d-bcc4-6ed72087dd45 /usr/sbin/ipadm show-addr -p -o ADDR oxControlService9/omicron6] executed and failed with status: exit status: 1 stdout: \n stderr: ipadm: Address object not found file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:841 latency_us = 26074 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::1]:53898 req_id = ae6f8b3b-4ffa-4b53-83ab-c868a869626a response_code = 500 uri = /cockroachdb
6662023-12-06T17:41:41.610ZWARNSledAgent (RSS): Failed to initialize CockroachDB error = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "ae6f8b3b-4ffa-4b53-83ab-c868a869626a", "content-length": "124", "date": "Wed, 06 Dec 2023 17:41:40 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "ae6f8b3b-4ffa-4b53-83ab-c868a869626a" } file = sled-agent/src/rack_setup/service.rs:758 retry_after = 241.665469ms
6672023-12-06T17:41:41.881ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = Failed to get address: Failed to get address for name oxControlService9/omicron6 in oxz_cockroachdb_5c732813-f138-434d-bcc4-6ed72087dd45: Zone execution error: Command [/usr/sbin/zlogin oxz_cockroachdb_5c732813-f138-434d-bcc4-6ed72087dd45 /usr/sbin/ipadm show-addr -p -o ADDR oxControlService9/omicron6] executed and failed with status: exit status: 1 stdout: \n stderr: ipadm: Address object not found file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:841 latency_us = 27649 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::1]:53898 req_id = 4da4750a-68a8-4903-988f-4fd81bf58421 response_code = 500 uri = /cockroachdb
6682023-12-06T17:41:41.881ZWARNSledAgent (RSS): Failed to initialize CockroachDB error = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "4da4750a-68a8-4903-988f-4fd81bf58421", "content-length": "124", "date": "Wed, 06 Dec 2023 17:41:40 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "4da4750a-68a8-4903-988f-4fd81bf58421" } file = sled-agent/src/rack_setup/service.rs:758 retry_after = 229.69229ms
6692023-12-06T17:41:42.141ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = Failed to get address: Failed to get address for name oxControlService9/omicron6 in oxz_cockroachdb_5c732813-f138-434d-bcc4-6ed72087dd45: Zone execution error: Command [/usr/sbin/zlogin oxz_cockroachdb_5c732813-f138-434d-bcc4-6ed72087dd45 /usr/sbin/ipadm show-addr -p -o ADDR oxControlService9/omicron6] executed and failed with status: exit status: 1 stdout: \n stderr: ipadm: Address object not found file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:841 latency_us = 29274 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::1]:53898 req_id = a7585a22-48f3-401b-84a1-57d9a31a2cc4 response_code = 500 uri = /cockroachdb
6702023-12-06T17:41:42.141ZWARNSledAgent (RSS): Failed to initialize CockroachDB error = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "a7585a22-48f3-401b-84a1-57d9a31a2cc4", "content-length": "124", "date": "Wed, 06 Dec 2023 17:41:42 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "a7585a22-48f3-401b-84a1-57d9a31a2cc4" } file = sled-agent/src/rack_setup/service.rs:758 retry_after = 422.658373ms
6712023-12-06T17:41:42.592ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = Failed to get address: Failed to get address for name oxControlService9/omicron6 in oxz_cockroachdb_5c732813-f138-434d-bcc4-6ed72087dd45: Zone execution error: Command [/usr/sbin/zlogin oxz_cockroachdb_5c732813-f138-434d-bcc4-6ed72087dd45 /usr/sbin/ipadm show-addr -p -o ADDR oxControlService9/omicron6] executed and failed with status: exit status: 1 stdout: \n stderr: ipadm: Address object not found file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:841 latency_us = 26492 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::1]:53898 req_id = d0e0c5b9-6b27-462e-9f66-6ef9105dcb9e response_code = 500 uri = /cockroachdb
6722023-12-06T17:41:42.592ZWARNSledAgent (RSS): Failed to initialize CockroachDB error = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "d0e0c5b9-6b27-462e-9f66-6ef9105dcb9e", "content-length": "124", "date": "Wed, 06 Dec 2023 17:41:42 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "d0e0c5b9-6b27-462e-9f66-6ef9105dcb9e" } file = sled-agent/src/rack_setup/service.rs:758 retry_after = 491.697995ms
6732023-12-06T17:41:43.112ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = Failed to get address: Failed to get address for name oxControlService9/omicron6 in oxz_cockroachdb_5c732813-f138-434d-bcc4-6ed72087dd45: Zone execution error: Command [/usr/sbin/zlogin oxz_cockroachdb_5c732813-f138-434d-bcc4-6ed72087dd45 /usr/sbin/ipadm show-addr -p -o ADDR oxControlService9/omicron6] executed and failed with status: exit status: 1 stdout: \n stderr: ipadm: Address object not found file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:841 latency_us = 27144 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::1]:53898 req_id = e9a77d98-48c8-4cad-ae63-ec0a6fef83b7 response_code = 500 uri = /cockroachdb
6742023-12-06T17:41:43.113ZWARNSledAgent (RSS): Failed to initialize CockroachDB error = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "e9a77d98-48c8-4cad-ae63-ec0a6fef83b7", "content-length": "124", "date": "Wed, 06 Dec 2023 17:41:43 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "e9a77d98-48c8-4cad-ae63-ec0a6fef83b7" } file = sled-agent/src/rack_setup/service.rs:758 retry_after = 578.726548ms
6752023-12-06T17:41:43.722ZINFOSledAgent (ServiceManager): Initializing CRDB Cluster - sending request to [fd00:1122:3344:101::6]:32221 file = sled-agent/src/services.rs:2881
6762023-12-06T17:41:51.503ZINFOSledAgent (ServiceManager): Formatting CRDB file = sled-agent/src/services.rs:2899
6772023-12-06T17:41:54.234ZINFOSledAgent (ServiceManager): Formatting CRDB - Completed file = sled-agent/src/services.rs:2920
6782023-12-06T17:41:54.235ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 10541665 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::1]:53898 req_id = eedc6de0-b20d-480b-88f6-ced39e7ad2ee response_code = 204 uri = /cockroachdb
6792023-12-06T17:41:54.236ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::60fc:8ff:fe95:bcc8": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
6802023-12-06T17:41:54.269ZINFOSledAgent (RSS): attempting to set up sled's Omicron zones: OmicronZonesConfig { generation: Generation(5), zones: [OmicronZoneConfig { id: 00f82355-ab6a-44f8-adeb-55a0298e7676, underlay_address: fd00:1122:3344:1::1, zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 14b4dc87-ab46-49fb-a4b4-d361ae214c03, kind: External } }, http_address: [fd00:1122:3344:1::1]:5353, dns_address: [fd00:1122:3344:1::1]:53, gz_address: fd00:1122:3344:1::2, gz_address_index: 0 } }, OmicronZoneConfig { id: be8d8c76-3f0a-4bd9-8466-47f0b72402ea, underlay_address: fd00:1122:3344:2::1, zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 24b4dc87-ab46-49fb-a4b4-d361ae214c03, kind: External } }, http_address: [fd00:1122:3344:2::1]:5353, dns_address: [fd00:1122:3344:2::1]:53, gz_address: fd00:1122:3344:2::2, gz_address_index: 1 } }, OmicronZoneConfig { id: ac3d1180-e1c3-477d-a802-3f8bd92e2528, underlay_address: fd00:1122:3344:3::1, zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 31bd71cd-4736-4a12-a387-9b74b050396f, kind: External } }, http_address: [fd00:1122:3344:3::1]:5353, dns_address: [fd00:1122:3344:3::1]:53, gz_address: fd00:1122:3344:3::2, gz_address_index: 2 } }, OmicronZoneConfig { id: 0e075832-9eaa-40c5-a328-dcde3812ee47, underlay_address: fd00:1122:3344:101::1b, zone_type: BoundaryNtp { address: [fd00:1122:3344:101::1b]:123, ntp_servers: ["0.pool.ntp.org"], dns_servers: [1.1.1.1, 9.9.9.9], domain: None, nic: NetworkInterface { id: 1277989f-b18b-4c6b-8166-0b015ebc20df, kind: Service { id: 0e075832-9eaa-40c5-a328-dcde3812ee47 }, name: Name("ntp-0e075832-9eaa-40c5-a328-dcde3812ee47"), ip: 172.30.3.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 143, 64])), subnet: V4(Ipv4Net(Ipv4Network { addr: 172.30.3.0, prefix: 24 })), vni: Vni(100), primary: true, slot: 0 }, snat_cfg: SourceNatConfig { ip: 10.151.2.169, first_port: 0, last_port: 16383 } } }, OmicronZoneConfig { id: ccbe62b8-1461-4dec-a1a5-93b3a022681d, underlay_address: fd00:1122:3344:101::3, zone_type: CockroachDb { address: [fd00:1122:3344:101::3]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 14b4dc87-ab46-49fb-a4b4-d361ae214c03, kind: External } } } }, OmicronZoneConfig { id: c9428da0-985f-4421-8de9-d60f987ba8ab, underlay_address: fd00:1122:3344:101::4, zone_type: CockroachDb { address: [fd00:1122:3344:101::4]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 24b4dc87-ab46-49fb-a4b4-d361ae214c03, kind: External } } } }, OmicronZoneConfig { id: 8f3a1d19-d2a3-45ba-b935-68f0390bcdf8, underlay_address: fd00:1122:3344:101::5, zone_type: CockroachDb { address: [fd00:1122:3344:101::5]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 31bd71cd-4736-4a12-a387-9b74b050396f, kind: External } } } }, OmicronZoneConfig { id: 5c732813-f138-434d-bcc4-6ed72087dd45, underlay_address: fd00:1122:3344:101::6, zone_type: CockroachDb { address: [fd00:1122:3344:101::6]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 616b26df-e62a-4c68-b506-f4a923d8aaf7, kind: External } } } }, OmicronZoneConfig { id: 8caca53a-45ee-4bbf-9f3b-bf2a9c9243ef, underlay_address: fd00:1122:3344:101::7, zone_type: CockroachDb { address: [fd00:1122:3344:101::7]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: cd70d7f6-2354-4bf2-8012-55bf9eaf7930, kind: External } } } }, OmicronZoneConfig { id: 2cfabf1e-5b74-40a8-8d00-5f984444bf51, underlay_address: fd00:1122:3344:101::8, zone_type: ExternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 14b4dc87-ab46-49fb-a4b4-d361ae214c03, kind: External } }, http_address: [fd00:1122:3344:101::8]:5353, dns_address: 10.151.2.164:53, nic: NetworkInterface { id: 58e4c4d4-d4dc-499f-91ef-477be3fc8fbf, kind: Service { id: 2cfabf1e-5b74-40a8-8d00-5f984444bf51 }, name: Name("external-dns-2cfabf1e-5b74-40a8-8d00-5f984444bf51"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 196, 173])), subnet: V4(Ipv4Net(Ipv4Network { addr: 172.30.1.0, prefix: 24 })), vni: Vni(100), primary: true, slot: 0 } } }, OmicronZoneConfig { id: f4bd4239-a995-46ad-bd81-8c0c7f2f9049, underlay_address: fd00:1122:3344:101::9, zone_type: ExternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 24b4dc87-ab46-49fb-a4b4-d361ae214c03, kind: External } }, http_address: [fd00:1122:3344:101::9]:5353, dns_address: 10.151.2.165:53, nic: NetworkInterface { id: 3a6a2032-caca-49e5-b81c-96d8b3f5cbfe, kind: Service { id: f4bd4239-a995-46ad-bd81-8c0c7f2f9049 }, name: Name("external-dns-f4bd4239-a995-46ad-bd81-8c0c7f2f9049"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 210, 64])), subnet: V4(Ipv4Net(Ipv4Network { addr: 172.30.1.0, prefix: 24 })), vni: Vni(100), primary: true, slot: 0 } } }, OmicronZoneConfig { id: d2016ddd-0efc-4439-89aa-2f166d289e6f, underlay_address: fd00:1122:3344:101::a, zone_type: Nexus { internal_address: [fd00:1122:3344:101::a]:12221, external_ip: 10.151.2.166, nic: NetworkInterface { id: 43f4b26a-16ba-4087-909c-b7a0873aa264, kind: Service { id: d2016ddd-0efc-4439-89aa-2f166d289e6f }, name: Name("nexus-d2016ddd-0efc-4439-89aa-2f166d289e6f"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 141, 175])), subnet: V4(Ipv4Net(Ipv4Network { addr: 172.30.2.0, prefix: 24 })), vni: Vni(100), primary: true, slot: 0 }, external_tls: true, external_dns_servers: [1.1.1.1, 9.9.9.9] } }, OmicronZoneConfig { id: 9a25727f-e844-4dd1-b1b4-978eca7f5c90, underlay_address: fd00:1122:3344:101::b, zone_type: Nexus { internal_address: [fd00:1122:3344:101::b]:12221, external_ip: 10.151.2.167, nic: NetworkInterface { id: 5465fc8d-98dc-4a8e-8ee0-96d7fdc4ec73, kind: Service { id: 9a25727f-e844-4dd1-b1b4-978eca7f5c90 }, name: Name("nexus-9a25727f-e844-4dd1-b1b4-978eca7f5c90"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 156, 174])), subnet: V4(Ipv4Net(Ipv4Network { addr: 172.30.2.0, prefix: 24 })), vni: Vni(100), primary: true, slot: 0 }, external_tls: true, external_dns_servers: [1.1.1.1, 9.9.9.9] } }, OmicronZoneConfig { id: 1dfcdbef-d660-4977-8363-0facffed5dd7, underlay_address: fd00:1122:3344:101::c, zone_type: Nexus { internal_address: [fd00:1122:3344:101::c]:12221, external_ip: 10.151.2.168, nic: NetworkInterface { id: 757a1ebf-4b4a-4ea0-a6c4-be5a8336ba90, kind: Service { id: 1dfcdbef-d660-4977-8363-0facffed5dd7 }, name: Name("nexus-1dfcdbef-d660-4977-8363-0facffed5dd7"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 248, 89])), subnet: V4(Ipv4Net(Ipv4Network { addr: 172.30.2.0, prefix: 24 })), vni: Vni(100), primary: true, slot: 0 }, external_tls: true, external_dns_servers: [1.1.1.1, 9.9.9.9] } }, OmicronZoneConfig { id: 358b59f5-3c51-4c63-92f1-1dcada1dc798, underlay_address: fd00:1122:3344:101::d, zone_type: Oximeter { address: [fd00:1122:3344:101::d]:12223 } }, OmicronZoneConfig { id: 345211c8-e540-44f1-be03-4dfe81553f49, underlay_address: fd00:1122:3344:101::e, zone_type: Clickhouse { address: [fd00:1122:3344:101::e]:8123, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 14b4dc87-ab46-49fb-a4b4-d361ae214c03, kind: External } } } }, OmicronZoneConfig { id: f6447f87-90dd-43a9-ac51-9e1831c10c3d, underlay_address: fd00:1122:3344:101::f, zone_type: CruciblePantry { address: [fd00:1122:3344:101::f]:17000 } }, OmicronZoneConfig { id: 5f32b190-8672-494f-90a3-7cd7719c5987, underlay_address: fd00:1122:3344:101::10, zone_type: CruciblePantry { address: [fd00:1122:3344:101::10]:17000 } }, OmicronZoneConfig { id: 0bfeea29-a47a-4323-893a-53a06601ad8e, underlay_address: fd00:1122:3344:101::11, zone_type: CruciblePantry { address: [fd00:1122:3344:101::11]:17000 } }, OmicronZoneConfig { id: 993ea282-1eee-42e3-b3d8-ea8aa4c06162, underlay_address: fd00:1122:3344:101::12, zone_type: Crucible { address: [fd00:1122:3344:101::12]:32345, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 14b4dc87-ab46-49fb-a4b4-d361ae214c03, kind: External } } } }, OmicronZoneConfig { id: 873ec7ab-6024-4863-b381-a4fcad4788ec, underlay_address: fd00:1122:3344:101::13, zone_type: Crucible { address: [fd00:1122:3344:101::13]:32345, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 24b4dc87-ab46-49fb-a4b4-d361ae214c03, kind: External } } } }, OmicronZoneConfig { id: 9bbb87d1-f50a-41e0-85ed-19784220ea16, underlay_address: fd00:1122:3344:101::14, zone_type: Crucible { address: [fd00:1122:3344:101::14]:32345, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 31bd71cd-4736-4a12-a387-9b74b050396f, kind: External } } } }, OmicronZoneConfig { id: f92ea1e7-347c-41d1-bd44-a15ef90d284c, underlay_address: fd00:1122:3344:101::15, zone_type: Crucible { address: [fd00:1122:3344:101::15]:32345, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 616b26df-e62a-4c68-b506-f4a923d8aaf7, kind: External } } } }, OmicronZoneConfig { id: 879f5844-21f5-4128-accd-2686d45ad2ab, underlay_address: fd00:1122:3344:101::16, zone_type: Crucible { address: [fd00:1122:3344:101::16]:32345, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: cd70d7f6-2354-4bf2-8012-55bf9eaf7930, kind: External } } } }, OmicronZoneConfig { id: f148960a-0913-4686-9975-ff2972d55de0, underlay_address: fd00:1122:3344:101::17, zone_type: Crucible { address: [fd00:1122:3344:101::17]:32345, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: ceb4461c-cf56-4719-ad3c-14430bfdfb60, kind: External } } } }, OmicronZoneConfig { id: 51de6a17-3c29-4573-9303-268fbfac8ee2, underlay_address: fd00:1122:3344:101::18, zone_type: Crucible { address: [fd00:1122:3344:101::18]:32345, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: d462a7f7-b628-40fe-80ff-4e4189e2d62b, kind: External } } } }, OmicronZoneConfig { id: 65a5c94d-7dd3-419e-b0c4-23533a58135d, underlay_address: fd00:1122:3344:101::19, zone_type: Crucible { address: [fd00:1122:3344:101::19]:32345, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: e4b4dc87-ab46-49fb-a4b4-d361ae214c03, kind: External } } } }, OmicronZoneConfig { id: f70a9b4a-82f3-4f7e-9b2e-db724f71ab50, underlay_address: fd00:1122:3344:101::1a, zone_type: Crucible { address: [fd00:1122:3344:101::1a]:32345, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: f4b4dc87-ab46-49fb-a4b4-d361ae214c03, kind: External } } } }] } file = sled-agent/src/rack_setup/service.rs:299 sled_address = [fd00:1122:3344:101::1]:12345
6812023-12-06T17:41:54.270ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:769 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:51442
6822023-12-06T17:41:54.271ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 00f82355-ab6a-44f8-adeb-55a0298e7676, dataset_name: DatasetName { pool_name: ZpoolName { id: 14b4dc87-ab46-49fb-a4b4-d361ae214c03, kind: External }, kind: InternalDns }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } }) file = sled-storage/src/manager.rs:346
6832023-12-06T17:41:54.271ZINFOSledAgent (StorageManager): add_dataset: NewFilesystemRequest { dataset_id: 00f82355-ab6a-44f8-adeb-55a0298e7676, dataset_name: DatasetName { pool_name: ZpoolName { id: 14b4dc87-ab46-49fb-a4b4-d361ae214c03, kind: External }, kind: InternalDns }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } } file = sled-storage/src/manager.rs:564
6842023-12-06T17:41:54.294ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: be8d8c76-3f0a-4bd9-8466-47f0b72402ea, dataset_name: DatasetName { pool_name: ZpoolName { id: 24b4dc87-ab46-49fb-a4b4-d361ae214c03, kind: External }, kind: InternalDns }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } }) file = sled-storage/src/manager.rs:346
6852023-12-06T17:41:54.294ZINFOSledAgent (StorageManager): add_dataset: NewFilesystemRequest { dataset_id: be8d8c76-3f0a-4bd9-8466-47f0b72402ea, dataset_name: DatasetName { pool_name: ZpoolName { id: 24b4dc87-ab46-49fb-a4b4-d361ae214c03, kind: External }, kind: InternalDns }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } } file = sled-storage/src/manager.rs:564
6862023-12-06T17:41:54.316ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: ac3d1180-e1c3-477d-a802-3f8bd92e2528, dataset_name: DatasetName { pool_name: ZpoolName { id: 31bd71cd-4736-4a12-a387-9b74b050396f, kind: External }, kind: InternalDns }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } }) file = sled-storage/src/manager.rs:346
6872023-12-06T17:41:54.316ZINFOSledAgent (StorageManager): add_dataset: NewFilesystemRequest { dataset_id: ac3d1180-e1c3-477d-a802-3f8bd92e2528, dataset_name: DatasetName { pool_name: ZpoolName { id: 31bd71cd-4736-4a12-a387-9b74b050396f, kind: External }, kind: InternalDns }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } } file = sled-storage/src/manager.rs:564
6882023-12-06T17:41:54.339ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: ccbe62b8-1461-4dec-a1a5-93b3a022681d, dataset_name: DatasetName { pool_name: ZpoolName { id: 14b4dc87-ab46-49fb-a4b4-d361ae214c03, kind: External }, kind: CockroachDb }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } }) file = sled-storage/src/manager.rs:346
6892023-12-06T17:41:54.339ZINFOSledAgent (StorageManager): add_dataset: NewFilesystemRequest { dataset_id: ccbe62b8-1461-4dec-a1a5-93b3a022681d, dataset_name: DatasetName { pool_name: ZpoolName { id: 14b4dc87-ab46-49fb-a4b4-d361ae214c03, kind: External }, kind: CockroachDb }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } } file = sled-storage/src/manager.rs:564
6902023-12-06T17:41:54.359ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: c9428da0-985f-4421-8de9-d60f987ba8ab, dataset_name: DatasetName { pool_name: ZpoolName { id: 24b4dc87-ab46-49fb-a4b4-d361ae214c03, kind: External }, kind: CockroachDb }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } }) file = sled-storage/src/manager.rs:346
6912023-12-06T17:41:54.359ZINFOSledAgent (StorageManager): add_dataset: NewFilesystemRequest { dataset_id: c9428da0-985f-4421-8de9-d60f987ba8ab, dataset_name: DatasetName { pool_name: ZpoolName { id: 24b4dc87-ab46-49fb-a4b4-d361ae214c03, kind: External }, kind: CockroachDb }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } } file = sled-storage/src/manager.rs:564
6922023-12-06T17:41:54.380ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 8f3a1d19-d2a3-45ba-b935-68f0390bcdf8, dataset_name: DatasetName { pool_name: ZpoolName { id: 31bd71cd-4736-4a12-a387-9b74b050396f, kind: External }, kind: CockroachDb }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } }) file = sled-storage/src/manager.rs:346
6932023-12-06T17:41:54.380ZINFOSledAgent (StorageManager): add_dataset: NewFilesystemRequest { dataset_id: 8f3a1d19-d2a3-45ba-b935-68f0390bcdf8, dataset_name: DatasetName { pool_name: ZpoolName { id: 31bd71cd-4736-4a12-a387-9b74b050396f, kind: External }, kind: CockroachDb }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } } file = sled-storage/src/manager.rs:564
6942023-12-06T17:41:54.400ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 5c732813-f138-434d-bcc4-6ed72087dd45, dataset_name: DatasetName { pool_name: ZpoolName { id: 616b26df-e62a-4c68-b506-f4a923d8aaf7, kind: External }, kind: CockroachDb }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } }) file = sled-storage/src/manager.rs:346
6952023-12-06T17:41:54.400ZINFOSledAgent (StorageManager): add_dataset: NewFilesystemRequest { dataset_id: 5c732813-f138-434d-bcc4-6ed72087dd45, dataset_name: DatasetName { pool_name: ZpoolName { id: 616b26df-e62a-4c68-b506-f4a923d8aaf7, kind: External }, kind: CockroachDb }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } } file = sled-storage/src/manager.rs:564
6962023-12-06T17:41:54.420ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 8caca53a-45ee-4bbf-9f3b-bf2a9c9243ef, dataset_name: DatasetName { pool_name: ZpoolName { id: cd70d7f6-2354-4bf2-8012-55bf9eaf7930, kind: External }, kind: CockroachDb }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } }) file = sled-storage/src/manager.rs:346
6972023-12-06T17:41:54.420ZINFOSledAgent (StorageManager): add_dataset: NewFilesystemRequest { dataset_id: 8caca53a-45ee-4bbf-9f3b-bf2a9c9243ef, dataset_name: DatasetName { pool_name: ZpoolName { id: cd70d7f6-2354-4bf2-8012-55bf9eaf7930, kind: External }, kind: CockroachDb }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } } file = sled-storage/src/manager.rs:564
6982023-12-06T17:41:54.440ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 2cfabf1e-5b74-40a8-8d00-5f984444bf51, dataset_name: DatasetName { pool_name: ZpoolName { id: 14b4dc87-ab46-49fb-a4b4-d361ae214c03, kind: External }, kind: ExternalDns }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } }) file = sled-storage/src/manager.rs:346
6992023-12-06T17:41:54.440ZINFOSledAgent (StorageManager): add_dataset: NewFilesystemRequest { dataset_id: 2cfabf1e-5b74-40a8-8d00-5f984444bf51, dataset_name: DatasetName { pool_name: ZpoolName { id: 14b4dc87-ab46-49fb-a4b4-d361ae214c03, kind: External }, kind: ExternalDns }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } } file = sled-storage/src/manager.rs:564
7002023-12-06T17:41:54.504ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: f4bd4239-a995-46ad-bd81-8c0c7f2f9049, dataset_name: DatasetName { pool_name: ZpoolName { id: 24b4dc87-ab46-49fb-a4b4-d361ae214c03, kind: External }, kind: ExternalDns }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } }) file = sled-storage/src/manager.rs:346
7012023-12-06T17:41:54.504ZINFOSledAgent (StorageManager): add_dataset: NewFilesystemRequest { dataset_id: f4bd4239-a995-46ad-bd81-8c0c7f2f9049, dataset_name: DatasetName { pool_name: ZpoolName { id: 24b4dc87-ab46-49fb-a4b4-d361ae214c03, kind: External }, kind: ExternalDns }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } } file = sled-storage/src/manager.rs:564
7022023-12-06T17:41:54.575ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 345211c8-e540-44f1-be03-4dfe81553f49, dataset_name: DatasetName { pool_name: ZpoolName { id: 14b4dc87-ab46-49fb-a4b4-d361ae214c03, kind: External }, kind: Clickhouse }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } }) file = sled-storage/src/manager.rs:346
7032023-12-06T17:41:54.575ZINFOSledAgent (StorageManager): add_dataset: NewFilesystemRequest { dataset_id: 345211c8-e540-44f1-be03-4dfe81553f49, dataset_name: DatasetName { pool_name: ZpoolName { id: 14b4dc87-ab46-49fb-a4b4-d361ae214c03, kind: External }, kind: Clickhouse }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } } file = sled-storage/src/manager.rs:564
7042023-12-06T17:41:54.635ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 993ea282-1eee-42e3-b3d8-ea8aa4c06162, dataset_name: DatasetName { pool_name: ZpoolName { id: 14b4dc87-ab46-49fb-a4b4-d361ae214c03, kind: External }, kind: Crucible }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } }) file = sled-storage/src/manager.rs:346
7052023-12-06T17:41:54.635ZINFOSledAgent (StorageManager): add_dataset: NewFilesystemRequest { dataset_id: 993ea282-1eee-42e3-b3d8-ea8aa4c06162, dataset_name: DatasetName { pool_name: ZpoolName { id: 14b4dc87-ab46-49fb-a4b4-d361ae214c03, kind: External }, kind: Crucible }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } } file = sled-storage/src/manager.rs:564
7062023-12-06T17:41:54.692ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 873ec7ab-6024-4863-b381-a4fcad4788ec, dataset_name: DatasetName { pool_name: ZpoolName { id: 24b4dc87-ab46-49fb-a4b4-d361ae214c03, kind: External }, kind: Crucible }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } }) file = sled-storage/src/manager.rs:346
7072023-12-06T17:41:54.692ZINFOSledAgent (StorageManager): add_dataset: NewFilesystemRequest { dataset_id: 873ec7ab-6024-4863-b381-a4fcad4788ec, dataset_name: DatasetName { pool_name: ZpoolName { id: 24b4dc87-ab46-49fb-a4b4-d361ae214c03, kind: External }, kind: Crucible }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } } file = sled-storage/src/manager.rs:564
7082023-12-06T17:41:54.759ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 9bbb87d1-f50a-41e0-85ed-19784220ea16, dataset_name: DatasetName { pool_name: ZpoolName { id: 31bd71cd-4736-4a12-a387-9b74b050396f, kind: External }, kind: Crucible }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } }) file = sled-storage/src/manager.rs:346
7092023-12-06T17:41:54.759ZINFOSledAgent (StorageManager): add_dataset: NewFilesystemRequest { dataset_id: 9bbb87d1-f50a-41e0-85ed-19784220ea16, dataset_name: DatasetName { pool_name: ZpoolName { id: 31bd71cd-4736-4a12-a387-9b74b050396f, kind: External }, kind: Crucible }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } } file = sled-storage/src/manager.rs:564
7102023-12-06T17:41:54.835ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: f92ea1e7-347c-41d1-bd44-a15ef90d284c, dataset_name: DatasetName { pool_name: ZpoolName { id: 616b26df-e62a-4c68-b506-f4a923d8aaf7, kind: External }, kind: Crucible }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } }) file = sled-storage/src/manager.rs:346
7112023-12-06T17:41:54.837ZINFOSledAgent (StorageManager): add_dataset: NewFilesystemRequest { dataset_id: f92ea1e7-347c-41d1-bd44-a15ef90d284c, dataset_name: DatasetName { pool_name: ZpoolName { id: 616b26df-e62a-4c68-b506-f4a923d8aaf7, kind: External }, kind: Crucible }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } } file = sled-storage/src/manager.rs:564
7122023-12-06T17:41:54.912ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 879f5844-21f5-4128-accd-2686d45ad2ab, dataset_name: DatasetName { pool_name: ZpoolName { id: cd70d7f6-2354-4bf2-8012-55bf9eaf7930, kind: External }, kind: Crucible }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } }) file = sled-storage/src/manager.rs:346
7132023-12-06T17:41:54.912ZINFOSledAgent (StorageManager): add_dataset: NewFilesystemRequest { dataset_id: 879f5844-21f5-4128-accd-2686d45ad2ab, dataset_name: DatasetName { pool_name: ZpoolName { id: cd70d7f6-2354-4bf2-8012-55bf9eaf7930, kind: External }, kind: Crucible }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } } file = sled-storage/src/manager.rs:564
7142023-12-06T17:41:54.989ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: f148960a-0913-4686-9975-ff2972d55de0, dataset_name: DatasetName { pool_name: ZpoolName { id: ceb4461c-cf56-4719-ad3c-14430bfdfb60, kind: External }, kind: Crucible }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } }) file = sled-storage/src/manager.rs:346
7152023-12-06T17:41:54.989ZINFOSledAgent (StorageManager): add_dataset: NewFilesystemRequest { dataset_id: f148960a-0913-4686-9975-ff2972d55de0, dataset_name: DatasetName { pool_name: ZpoolName { id: ceb4461c-cf56-4719-ad3c-14430bfdfb60, kind: External }, kind: Crucible }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } } file = sled-storage/src/manager.rs:564
7162023-12-06T17:41:55.061ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 51de6a17-3c29-4573-9303-268fbfac8ee2, dataset_name: DatasetName { pool_name: ZpoolName { id: d462a7f7-b628-40fe-80ff-4e4189e2d62b, kind: External }, kind: Crucible }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } }) file = sled-storage/src/manager.rs:346
7172023-12-06T17:41:55.061ZINFOSledAgent (StorageManager): add_dataset: NewFilesystemRequest { dataset_id: 51de6a17-3c29-4573-9303-268fbfac8ee2, dataset_name: DatasetName { pool_name: ZpoolName { id: d462a7f7-b628-40fe-80ff-4e4189e2d62b, kind: External }, kind: Crucible }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } } file = sled-storage/src/manager.rs:564
7182023-12-06T17:41:55.129ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 65a5c94d-7dd3-419e-b0c4-23533a58135d, dataset_name: DatasetName { pool_name: ZpoolName { id: e4b4dc87-ab46-49fb-a4b4-d361ae214c03, kind: External }, kind: Crucible }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } }) file = sled-storage/src/manager.rs:346
7192023-12-06T17:41:55.129ZINFOSledAgent (StorageManager): add_dataset: NewFilesystemRequest { dataset_id: 65a5c94d-7dd3-419e-b0c4-23533a58135d, dataset_name: DatasetName { pool_name: ZpoolName { id: e4b4dc87-ab46-49fb-a4b4-d361ae214c03, kind: External }, kind: Crucible }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } } file = sled-storage/src/manager.rs:564
7202023-12-06T17:41:55.198ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: f70a9b4a-82f3-4f7e-9b2e-db724f71ab50, dataset_name: DatasetName { pool_name: ZpoolName { id: f4b4dc87-ab46-49fb-a4b4-d361ae214c03, kind: External }, kind: Crucible }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } }) file = sled-storage/src/manager.rs:346
7212023-12-06T17:41:55.198ZINFOSledAgent (StorageManager): add_dataset: NewFilesystemRequest { dataset_id: f70a9b4a-82f3-4f7e-9b2e-db724f71ab50, dataset_name: DatasetName { pool_name: ZpoolName { id: f4b4dc87-ab46-49fb-a4b4-d361ae214c03, kind: External }, kind: Crucible }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } } file = sled-storage/src/manager.rs:564
7222023-12-06T17:41:55.277ZINFOSledAgent (StorageManager): Received GetLatestResources(Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) }) file = sled-storage/src/manager.rs:346
7232023-12-06T17:41:55.277ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/a462a7f7-b628-40fe-80ff-4e4189e2d62b/config/omicron-zones.json file = common/src/ledger.rs:176
7242023-12-06T17:41:55.278ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/b462a7f7-b628-40fe-80ff-4e4189e2d62b/config/omicron-zones.json file = common/src/ledger.rs:176
7252023-12-06T17:41:55.278ZINFOSledAgent (StorageManager): Received GetLatestResources(Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) }) file = sled-storage/src/manager.rs:346
7262023-12-06T17:41:55.284ZINFOSledAgent (ServiceManager): Resolving switch zone addresses in DNS file = sled-agent/src/bootstrap/early_networking.rs:253
7272023-12-06T17:41:55.285ZINFOSledAgent (StorageManager): Received GetLatestResources(Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) }) file = sled-storage/src/manager.rs:346
7282023-12-06T17:41:55.285ZINFOSledAgent (StorageManager): Received GetLatestResources(Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) }) file = sled-storage/src/manager.rs:346
7292023-12-06T17:41:55.285ZINFOSledAgent (StorageManager): Received GetLatestResources(Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) }) file = sled-storage/src/manager.rs:346
7302023-12-06T17:41:55.289ZINFOSledAgent (ServiceManager): Resolving switch zone addresses in DNS file = sled-agent/src/bootstrap/early_networking.rs:253
7312023-12-06T17:41:55.290ZINFOSledAgent (StorageManager): Received GetLatestResources(Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) }) file = sled-storage/src/manager.rs:346
7322023-12-06T17:41:55.290ZINFOSledAgent (StorageManager): Received GetLatestResources(Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) }) file = sled-storage/src/manager.rs:346
7332023-12-06T17:41:55.290ZINFOSledAgent (StorageManager): Received GetLatestResources(Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) }) file = sled-storage/src/manager.rs:346
7342023-12-06T17:41:55.292ZINFOSledAgent (ServiceManager): Resolving switch zone addresses in DNS file = sled-agent/src/bootstrap/early_networking.rs:253
7352023-12-06T17:41:55.292ZINFOSledAgent (StorageManager): Received GetLatestResources(Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) }) file = sled-storage/src/manager.rs:346
7362023-12-06T17:41:55.295ZINFOSledAgent (ServiceManager): Resolving switch zone addresses in DNS file = sled-agent/src/bootstrap/early_networking.rs:253
7372023-12-06T17:41:55.295ZINFOSledAgent (StorageManager): Received GetLatestResources(Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) }) file = sled-storage/src/manager.rs:346
7382023-12-06T17:41:55.302ZINFOSledAgent (ServiceManager): Resolving switch zone addresses in DNS file = sled-agent/src/bootstrap/early_networking.rs:253
7392023-12-06T17:41:55.302ZINFOSledAgent (StorageManager): Received GetLatestResources(Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) }) file = sled-storage/src/manager.rs:346
7402023-12-06T17:41:55.302ZINFOSledAgent (StorageManager): Received GetLatestResources(Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) }) file = sled-storage/src/manager.rs:346
7412023-12-06T17:41:55.302ZINFOSledAgent (StorageManager): Received GetLatestResources(Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) }) file = sled-storage/src/manager.rs:346
7422023-12-06T17:41:55.302ZINFOSledAgent (StorageManager): Received GetLatestResources(Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) }) file = sled-storage/src/manager.rs:346
7432023-12-06T17:41:55.302ZINFOSledAgent (StorageManager): Received GetLatestResources(Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) }) file = sled-storage/src/manager.rs:346
7442023-12-06T17:41:55.304ZINFOSledAgent (StorageManager): Received GetLatestResources(Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) }) file = sled-storage/src/manager.rs:346
7452023-12-06T17:41:55.607ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_clickhouse_345211c8-e540-44f1-be03-4dfe81553f49 file = illumos-utils/src/zone.rs:314
7462023-12-06T17:41:55.612ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_65a5c94d-7dd3-419e-b0c4-23533a58135d file = illumos-utils/src/zone.rs:314
7472023-12-06T17:41:55.617ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_f148960a-0913-4686-9975-ff2972d55de0 file = illumos-utils/src/zone.rs:314
7482023-12-06T17:41:55.622ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_pantry_0bfeea29-a47a-4323-893a-53a06601ad8e file = illumos-utils/src/zone.rs:314
7492023-12-06T17:41:55.627ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_9bbb87d1-f50a-41e0-85ed-19784220ea16 file = illumos-utils/src/zone.rs:314
7502023-12-06T17:41:55.633ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_f92ea1e7-347c-41d1-bd44-a15ef90d284c file = illumos-utils/src/zone.rs:314
7512023-12-06T17:41:55.638ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_oximeter_358b59f5-3c51-4c63-92f1-1dcada1dc798 file = illumos-utils/src/zone.rs:314
7522023-12-06T17:41:55.644ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_pantry_5f32b190-8672-494f-90a3-7cd7719c5987 file = illumos-utils/src/zone.rs:314
7532023-12-06T17:41:55.650ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_873ec7ab-6024-4863-b381-a4fcad4788ec file = illumos-utils/src/zone.rs:314
7542023-12-06T17:41:55.655ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_clickhouse_345211c8-e540-44f1-be03-4dfe81553f49 file = illumos-utils/src/zone.rs:353
7552023-12-06T17:41:55.661ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_65a5c94d-7dd3-419e-b0c4-23533a58135d file = illumos-utils/src/zone.rs:353
7562023-12-06T17:41:55.666ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_f148960a-0913-4686-9975-ff2972d55de0 file = illumos-utils/src/zone.rs:353
7572023-12-06T17:41:55.672ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_pantry_0bfeea29-a47a-4323-893a-53a06601ad8e file = illumos-utils/src/zone.rs:353
7582023-12-06T17:41:55.678ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_9bbb87d1-f50a-41e0-85ed-19784220ea16 file = illumos-utils/src/zone.rs:353
7592023-12-06T17:41:55.683ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_f92ea1e7-347c-41d1-bd44-a15ef90d284c file = illumos-utils/src/zone.rs:353
7602023-12-06T17:41:55.688ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_oximeter_358b59f5-3c51-4c63-92f1-1dcada1dc798 file = illumos-utils/src/zone.rs:353
7612023-12-06T17:41:55.694ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_pantry_5f32b190-8672-494f-90a3-7cd7719c5987 file = illumos-utils/src/zone.rs:353
7622023-12-06T17:41:55.700ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_873ec7ab-6024-4863-b381-a4fcad4788ec file = illumos-utils/src/zone.rs:353
7632023-12-06T17:41:55.706ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_879f5844-21f5-4128-accd-2686d45ad2ab file = illumos-utils/src/zone.rs:314
7642023-12-06T17:41:55.711ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_51de6a17-3c29-4573-9303-268fbfac8ee2 file = illumos-utils/src/zone.rs:314
7652023-12-06T17:41:55.716ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_f70a9b4a-82f3-4f7e-9b2e-db724f71ab50 file = illumos-utils/src/zone.rs:314
7662023-12-06T17:41:55.722ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_pantry_f6447f87-90dd-43a9-ac51-9e1831c10c3d file = illumos-utils/src/zone.rs:314
7672023-12-06T17:41:55.729ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_993ea282-1eee-42e3-b3d8-ea8aa4c06162 file = illumos-utils/src/zone.rs:314
7682023-12-06T17:41:55.737ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_879f5844-21f5-4128-accd-2686d45ad2ab file = illumos-utils/src/zone.rs:353
7692023-12-06T17:41:55.743ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_51de6a17-3c29-4573-9303-268fbfac8ee2 file = illumos-utils/src/zone.rs:353
7702023-12-06T17:41:55.749ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_f70a9b4a-82f3-4f7e-9b2e-db724f71ab50 file = illumos-utils/src/zone.rs:353
7712023-12-06T17:41:55.754ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_pantry_f6447f87-90dd-43a9-ac51-9e1831c10c3d file = illumos-utils/src/zone.rs:353
7722023-12-06T17:41:55.760ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_993ea282-1eee-42e3-b3d8-ea8aa4c06162 file = illumos-utils/src/zone.rs:353
7732023-12-06T17:41:55.808ZINFOSledAgent (ServiceManager): Querying MGS to determine switch location addr = fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:270
7742023-12-06T17:41:55.847ZINFOSledAgent (ServiceManager): Querying MGS to determine switch location addr = fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:270
7752023-12-06T17:41:55.882ZINFOSledAgent (ServiceManager): Querying MGS to determine switch location addr = fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:270
7762023-12-06T17:41:55.923ZINFOSledAgent (ServiceManager): Querying MGS to determine switch location addr = fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:270
7772023-12-06T17:41:55.968ZINFOSledAgent (ServiceManager): Querying MGS to determine switch location addr = fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:270
7782023-12-06T17:41:55.970ZINFOSledAgent (ServiceManager): Found Switch0 at fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:295
7792023-12-06T17:41:55.970ZINFOSledAgent (ServiceManager): Successfully looked up all expected switch zone underlay addresses addrs = {Switch0: fd00:1122:3344:101::2} file = sled-agent/src/bootstrap/early_networking.rs:187
7802023-12-06T17:41:56.026ZINFOSledAgent (PortManager): Created OPTE port file = illumos-utils/src/opte/port_manager.rs:383 port = Port { inner: PortInner { name: "opte1", _ip: 172.30.2.6, mac: MacAddr6([168, 64, 37, 255, 156, 174]), slot: 0, vni: Vni { inner: 100 }, gateway: Gateway { mac: MacAddr6([168, 64, 37, 255, 119, 119]), ip: 172.30.2.1 }, vnic: "vopte1" } }
7812023-12-06T17:41:56.026ZINFOSledAgent (ServiceManager): creating NAT entry for service file = sled-agent/src/services.rs:1226 zone_type = nexus
7822023-12-06T17:41:56.030ZINFOSledAgent (ServiceManager): Found Switch0 at fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:295
7832023-12-06T17:41:56.030ZINFOSledAgent (ServiceManager): Successfully looked up all expected switch zone underlay addresses addrs = {Switch0: fd00:1122:3344:101::2} file = sled-agent/src/bootstrap/early_networking.rs:187
7842023-12-06T17:41:56.083ZINFOSledAgent (PortManager): Created OPTE port file = illumos-utils/src/opte/port_manager.rs:383 port = Port { inner: PortInner { name: "opte2", _ip: 172.30.2.5, mac: MacAddr6([168, 64, 37, 255, 141, 175]), slot: 0, vni: Vni { inner: 100 }, gateway: Gateway { mac: MacAddr6([168, 64, 37, 255, 119, 119]), ip: 172.30.2.1 }, vnic: "vopte2" } }
7852023-12-06T17:41:56.083ZINFOSledAgent (ServiceManager): creating NAT entry for service file = sled-agent/src/services.rs:1226 zone_type = nexus
7862023-12-06T17:41:56.086ZINFOSledAgent (ServiceManager): Found Switch0 at fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:295
7872023-12-06T17:41:56.086ZINFOSledAgent (ServiceManager): Successfully looked up all expected switch zone underlay addresses addrs = {Switch0: fd00:1122:3344:101::2} file = sled-agent/src/bootstrap/early_networking.rs:187
7882023-12-06T17:41:56.140ZINFOSledAgent (PortManager): Created OPTE port file = illumos-utils/src/opte/port_manager.rs:383 port = Port { inner: PortInner { name: "opte3", _ip: 172.30.1.5, mac: MacAddr6([168, 64, 37, 255, 196, 173]), slot: 0, vni: Vni { inner: 100 }, gateway: Gateway { mac: MacAddr6([168, 64, 37, 255, 119, 119]), ip: 172.30.1.1 }, vnic: "vopte3" } }
7892023-12-06T17:41:56.140ZINFOSledAgent (ServiceManager): creating NAT entry for service file = sled-agent/src/services.rs:1226 zone_type = external_dns
7902023-12-06T17:41:56.141ZINFOSledAgent (ServiceManager): Found Switch0 at fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:295
7912023-12-06T17:41:56.141ZINFOSledAgent (ServiceManager): Successfully looked up all expected switch zone underlay addresses addrs = {Switch0: fd00:1122:3344:101::2} file = sled-agent/src/bootstrap/early_networking.rs:187
7922023-12-06T17:41:56.196ZINFOSledAgent (PortManager): Created OPTE port file = illumos-utils/src/opte/port_manager.rs:383 port = Port { inner: PortInner { name: "opte4", _ip: 172.30.1.6, mac: MacAddr6([168, 64, 37, 255, 210, 64]), slot: 0, vni: Vni { inner: 100 }, gateway: Gateway { mac: MacAddr6([168, 64, 37, 255, 119, 119]), ip: 172.30.1.1 }, vnic: "vopte4" } }
7932023-12-06T17:41:56.196ZINFOSledAgent (ServiceManager): creating NAT entry for service file = sled-agent/src/services.rs:1226 zone_type = external_dns
7942023-12-06T17:41:56.198ZINFOSledAgent (ServiceManager): Found Switch0 at fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:295
7952023-12-06T17:41:56.198ZINFOSledAgent (ServiceManager): Successfully looked up all expected switch zone underlay addresses addrs = {Switch0: fd00:1122:3344:101::2} file = sled-agent/src/bootstrap/early_networking.rs:187
7962023-12-06T17:41:56.259ZINFOSledAgent (PortManager): Created OPTE port file = illumos-utils/src/opte/port_manager.rs:383 port = Port { inner: PortInner { name: "opte5", _ip: 172.30.2.7, mac: MacAddr6([168, 64, 37, 255, 248, 89]), slot: 0, vni: Vni { inner: 100 }, gateway: Gateway { mac: MacAddr6([168, 64, 37, 255, 119, 119]), ip: 172.30.2.1 }, vnic: "vopte5" } }
7972023-12-06T17:41:56.259ZINFOSledAgent (ServiceManager): creating NAT entry for service file = sled-agent/src/services.rs:1226 zone_type = nexus
7982023-12-06T17:41:56.260ZINFOSledAgent (ServiceManager): no nat entry found for: V4( Ipv4Network { addr: 10.151.2.167, prefix: 32, }, ) file = clients/dpd-client/src/lib.rs:98
7992023-12-06T17:41:56.260ZINFOSledAgent (ServiceManager): creating nat entry for: V4( Ipv4Network { addr: 10.151.2.167, prefix: 32, }, ) file = clients/dpd-client/src/lib.rs:105
8002023-12-06T17:41:56.262ZINFOSledAgent (ServiceManager): no nat entry found for: V4( Ipv4Network { addr: 10.151.2.168, prefix: 32, }, ) file = clients/dpd-client/src/lib.rs:98
8012023-12-06T17:41:56.262ZINFOSledAgent (ServiceManager): creating nat entry for: V4( Ipv4Network { addr: 10.151.2.168, prefix: 32, }, ) file = clients/dpd-client/src/lib.rs:105
8022023-12-06T17:41:56.262ZINFOSledAgent (ServiceManager): no nat entry found for: V4( Ipv4Network { addr: 10.151.2.166, prefix: 32, }, ) file = clients/dpd-client/src/lib.rs:98
8032023-12-06T17:41:56.262ZINFOSledAgent (ServiceManager): creating nat entry for: V4( Ipv4Network { addr: 10.151.2.166, prefix: 32, }, ) file = clients/dpd-client/src/lib.rs:105
8042023-12-06T17:41:56.263ZINFOSledAgent (ServiceManager): no nat entry found for: V4( Ipv4Network { addr: 10.151.2.164, prefix: 32, }, ) file = clients/dpd-client/src/lib.rs:98
8052023-12-06T17:41:56.263ZINFOSledAgent (ServiceManager): creating nat entry for: V4( Ipv4Network { addr: 10.151.2.164, prefix: 32, }, ) file = clients/dpd-client/src/lib.rs:105
8062023-12-06T17:41:56.263ZINFOSledAgent (ServiceManager): no nat entry found for: V4( Ipv4Network { addr: 10.151.2.165, prefix: 32, }, ) file = clients/dpd-client/src/lib.rs:98
8072023-12-06T17:41:56.263ZINFOSledAgent (ServiceManager): creating nat entry for: V4( Ipv4Network { addr: 10.151.2.165, prefix: 32, }, ) file = clients/dpd-client/src/lib.rs:105
8082023-12-06T17:41:56.263ZINFOSledAgent (ServiceManager): creation of nat entry successful for: V4( Ipv4Network { addr: 10.151.2.167, prefix: 32, }, ) file = clients/dpd-client/src/lib.rs:133
8092023-12-06T17:41:56.267ZINFOSledAgent (ServiceManager): creation of nat entry successful for: V4( Ipv4Network { addr: 10.151.2.168, prefix: 32, }, ) file = clients/dpd-client/src/lib.rs:133
8102023-12-06T17:41:56.269ZINFOSledAgent (StorageManager): Received GetLatestResources(Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) }) file = sled-storage/src/manager.rs:346
8112023-12-06T17:41:56.269ZINFOSledAgent (StorageManager): Received GetLatestResources(Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) }) file = sled-storage/src/manager.rs:346
8122023-12-06T17:41:56.269ZINFOSledAgent (ServiceManager): creation of nat entry successful for: V4( Ipv4Network { addr: 10.151.2.166, prefix: 32, }, ) file = clients/dpd-client/src/lib.rs:133
8132023-12-06T17:41:56.271ZINFOSledAgent (ServiceManager): creation of nat entry successful for: V4( Ipv4Network { addr: 10.151.2.164, prefix: 32, }, ) file = clients/dpd-client/src/lib.rs:133
8142023-12-06T17:41:56.273ZINFOSledAgent (StorageManager): Received GetLatestResources(Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) }) file = sled-storage/src/manager.rs:346
8152023-12-06T17:41:56.273ZINFOSledAgent (StorageManager): Received GetLatestResources(Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) }) file = sled-storage/src/manager.rs:346
8162023-12-06T17:41:56.273ZINFOSledAgent (ServiceManager): creation of nat entry successful for: V4( Ipv4Network { addr: 10.151.2.165, prefix: 32, }, ) file = clients/dpd-client/src/lib.rs:133
8172023-12-06T17:41:56.302ZINFOSledAgent (StorageManager): Received GetLatestResources(Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) }) file = sled-storage/src/manager.rs:346
8182023-12-06T17:41:56.396ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_nexus_1dfcdbef-d660-4977-8363-0facffed5dd7 file = illumos-utils/src/zone.rs:314
8192023-12-06T17:41:56.401ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_nexus_9a25727f-e844-4dd1-b1b4-978eca7f5c90 file = illumos-utils/src/zone.rs:314
8202023-12-06T17:41:56.406ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_nexus_d2016ddd-0efc-4439-89aa-2f166d289e6f file = illumos-utils/src/zone.rs:314
8212023-12-06T17:41:56.412ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_external_dns_2cfabf1e-5b74-40a8-8d00-5f984444bf51 file = illumos-utils/src/zone.rs:314
8222023-12-06T17:41:56.418ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_external_dns_f4bd4239-a995-46ad-bd81-8c0c7f2f9049 file = illumos-utils/src/zone.rs:314
8232023-12-06T17:41:56.425ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_nexus_1dfcdbef-d660-4977-8363-0facffed5dd7 file = illumos-utils/src/zone.rs:353
8242023-12-06T17:41:56.431ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_nexus_9a25727f-e844-4dd1-b1b4-978eca7f5c90 file = illumos-utils/src/zone.rs:353
8252023-12-06T17:41:56.436ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_nexus_d2016ddd-0efc-4439-89aa-2f166d289e6f file = illumos-utils/src/zone.rs:353
8262023-12-06T17:41:56.442ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_external_dns_2cfabf1e-5b74-40a8-8d00-5f984444bf51 file = illumos-utils/src/zone.rs:353
8272023-12-06T17:41:56.447ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_external_dns_f4bd4239-a995-46ad-bd81-8c0c7f2f9049 file = illumos-utils/src/zone.rs:353
8282023-12-06T17:41:59.238ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::60fc:8ff:fe95:bcc8": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
8292023-12-06T17:41:59.896ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_pantry_f6447f87-90dd-43a9-ac51-9e1831c10c3d: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/crucible/pantry"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value="oxControlService22"/> <propval type="astring" name="gateway" value="fd00:1122:3344:101::1"/> <propval type="astring" name="listen_addr" value="fd00:1122:3344:101::f"/> <propval type="astring" name="listen_port" value="17000"/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
8302023-12-06T17:41:59.897ZINFOSledAgent (ServiceManager): Zone booting file = illumos-utils/src/running_zone.rs:512 zone = oxz_crucible_pantry_f6447f87-90dd-43a9-ac51-9e1831c10c3d
8312023-12-06T17:42:00.675ZINFOSledAgent (ServiceManager): Zone booting file = illumos-utils/src/running_zone.rs:512 zone = oxz_external_dns_f4bd4239-a995-46ad-bd81-8c0c7f2f9049
8322023-12-06T17:42:00.872ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_993ea282-1eee-42e3-b3d8-ea8aa4c06162: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/crucible/agent"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value="oxControlService23"/> <propval type="astring" name="gateway" value="fd00:1122:3344:101::1"/> <propval type="astring" name="dataset" value="oxp_14b4dc87-ab46-49fb-a4b4-d361ae214c03/crucible"/> <propval type="astring" name="listen_addr" value="fd00:1122:3344:101::12"/> <propval type="astring" name="listen_port" value="32345"/> <propval type="astring" name="uuid" value="be91a6ba-9488-434c-867c-fc553689a080"/> <propval type="astring" name="store" value="/data"/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
8332023-12-06T17:42:00.873ZINFOSledAgent (ServiceManager): Zone booting file = illumos-utils/src/running_zone.rs:512 zone = oxz_crucible_993ea282-1eee-42e3-b3d8-ea8aa4c06162
8342023-12-06T17:42:00.899ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_9bbb87d1-f50a-41e0-85ed-19784220ea16: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/crucible/agent"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value="oxControlService14"/> <propval type="astring" name="gateway" value="fd00:1122:3344:101::1"/> <propval type="astring" name="dataset" value="oxp_31bd71cd-4736-4a12-a387-9b74b050396f/crucible"/> <propval type="astring" name="listen_addr" value="fd00:1122:3344:101::14"/> <propval type="astring" name="listen_port" value="32345"/> <propval type="astring" name="uuid" value="f622fa6c-b8cc-47df-80bf-c86421789fea"/> <propval type="astring" name="store" value="/data"/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
8352023-12-06T17:42:00.899ZINFOSledAgent (ServiceManager): Zone booting file = illumos-utils/src/running_zone.rs:512 zone = oxz_crucible_9bbb87d1-f50a-41e0-85ed-19784220ea16
8362023-12-06T17:42:00.969ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_f6447f87-90dd-43a9-ac51-9e1831c10c3d") failed: Property not found. retry in 35.115309ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_f6447f87-90dd-43a9-ac51-9e1831c10c3d
8372023-12-06T17:42:01.011ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_f4bd4239-a995-46ad-bd81-8c0c7f2f9049") failed: Property not found. retry in 70.978597ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_f4bd4239-a995-46ad-bd81-8c0c7f2f9049
8382023-12-06T17:42:01.024ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_f6447f87-90dd-43a9-ac51-9e1831c10c3d") failed: Property not found. retry in 112.574928ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_f6447f87-90dd-43a9-ac51-9e1831c10c3d
8392023-12-06T17:42:01.025ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_51de6a17-3c29-4573-9303-268fbfac8ee2: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/crucible/agent"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value="oxControlService20"/> <propval type="astring" name="gateway" value="fd00:1122:3344:101::1"/> <propval type="astring" name="dataset" value="oxp_d462a7f7-b628-40fe-80ff-4e4189e2d62b/crucible"/> <propval type="astring" name="listen_addr" value="fd00:1122:3344:101::18"/> <propval type="astring" name="listen_port" value="32345"/> <propval type="astring" name="uuid" value="0372c3b9-fe01-4a01-a410-d9d101c0bd48"/> <propval type="astring" name="store" value="/data"/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
8402023-12-06T17:42:01.025ZINFOSledAgent (ServiceManager): Zone booting file = illumos-utils/src/running_zone.rs:512 zone = oxz_crucible_51de6a17-3c29-4573-9303-268fbfac8ee2
8412023-12-06T17:42:01.095ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_f4bd4239-a995-46ad-bd81-8c0c7f2f9049") failed: Property not found. retry in 93.401956ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_f4bd4239-a995-46ad-bd81-8c0c7f2f9049
8422023-12-06T17:42:01.108ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_993ea282-1eee-42e3-b3d8-ea8aa4c06162") failed: Property not found. retry in 50.77287ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_993ea282-1eee-42e3-b3d8-ea8aa4c06162
8432023-12-06T17:42:01.121ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_9bbb87d1-f50a-41e0-85ed-19784220ea16") failed: Property not found. retry in 27.469082ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_9bbb87d1-f50a-41e0-85ed-19784220ea16
8442023-12-06T17:42:01.156ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_f6447f87-90dd-43a9-ac51-9e1831c10c3d") failed: Property not found. retry in 228.657615ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_f6447f87-90dd-43a9-ac51-9e1831c10c3d
8452023-12-06T17:42:01.167ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_9bbb87d1-f50a-41e0-85ed-19784220ea16") failed: Property not found. retry in 83.370866ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_9bbb87d1-f50a-41e0-85ed-19784220ea16
8462023-12-06T17:42:01.179ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_993ea282-1eee-42e3-b3d8-ea8aa4c06162") failed: Property not found. retry in 98.859883ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_993ea282-1eee-42e3-b3d8-ea8aa4c06162
8472023-12-06T17:42:01.204ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_f4bd4239-a995-46ad-bd81-8c0c7f2f9049") failed: Property not found. retry in 192.915838ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_f4bd4239-a995-46ad-bd81-8c0c7f2f9049
8482023-12-06T17:42:01.261ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_9bbb87d1-f50a-41e0-85ed-19784220ea16") failed: Property not found. retry in 219.54828ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_9bbb87d1-f50a-41e0-85ed-19784220ea16
8492023-12-06T17:42:01.281ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_51de6a17-3c29-4573-9303-268fbfac8ee2") failed: Property not found. retry in 74.606242ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_51de6a17-3c29-4573-9303-268fbfac8ee2
8502023-12-06T17:42:01.291ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_993ea282-1eee-42e3-b3d8-ea8aa4c06162") failed: Property not found. retry in 190.428973ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_993ea282-1eee-42e3-b3d8-ea8aa4c06162
8512023-12-06T17:42:01.368ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_51de6a17-3c29-4573-9303-268fbfac8ee2") failed: Property not found. retry in 128.579469ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_51de6a17-3c29-4573-9303-268fbfac8ee2
8522023-12-06T17:42:01.397ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_f6447f87-90dd-43a9-ac51-9e1831c10c3d") failed: Property not found. retry in 505.728608ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_f6447f87-90dd-43a9-ac51-9e1831c10c3d
8532023-12-06T17:42:01.408ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_f4bd4239-a995-46ad-bd81-8c0c7f2f9049") failed: Property not found. retry in 325.965127ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_f4bd4239-a995-46ad-bd81-8c0c7f2f9049
8542023-12-06T17:42:01.495ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_9bbb87d1-f50a-41e0-85ed-19784220ea16") failed: Property not found. retry in 521.737346ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_9bbb87d1-f50a-41e0-85ed-19784220ea16
8552023-12-06T17:42:01.508ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_993ea282-1eee-42e3-b3d8-ea8aa4c06162") failed: Property not found. retry in 260.349406ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_993ea282-1eee-42e3-b3d8-ea8aa4c06162
8562023-12-06T17:42:01.521ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_51de6a17-3c29-4573-9303-268fbfac8ee2") failed: Property not found. retry in 190.394946ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_51de6a17-3c29-4573-9303-268fbfac8ee2
8572023-12-06T17:42:01.521ZINFOSledAgent (ServiceManager): Zone booting file = illumos-utils/src/running_zone.rs:512 zone = oxz_nexus_1dfcdbef-d660-4977-8363-0facffed5dd7
8582023-12-06T17:42:01.755ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_51de6a17-3c29-4573-9303-268fbfac8ee2") failed: Property not found. retry in 397.319562ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_51de6a17-3c29-4573-9303-268fbfac8ee2
8592023-12-06T17:42:01.827ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_f4bd4239-a995-46ad-bd81-8c0c7f2f9049") failed: Property not found. retry in 749.635677ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_f4bd4239-a995-46ad-bd81-8c0c7f2f9049
8602023-12-06T17:42:01.854ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_993ea282-1eee-42e3-b3d8-ea8aa4c06162") failed: Property not found. retry in 966.181323ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_993ea282-1eee-42e3-b3d8-ea8aa4c06162
8612023-12-06T17:42:01.936ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_f6447f87-90dd-43a9-ac51-9e1831c10c3d") failed: Property not found. retry in 736.784992ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_f6447f87-90dd-43a9-ac51-9e1831c10c3d
8622023-12-06T17:42:02.010ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_1dfcdbef-d660-4977-8363-0facffed5dd7") failed: Property not found. retry in 28.251248ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_1dfcdbef-d660-4977-8363-0facffed5dd7
8632023-12-06T17:42:02.035ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_9bbb87d1-f50a-41e0-85ed-19784220ea16") failed: Property not found. retry in 672.020277ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_9bbb87d1-f50a-41e0-85ed-19784220ea16
8642023-12-06T17:42:02.054ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_1dfcdbef-d660-4977-8363-0facffed5dd7") failed: Property not found. retry in 53.717186ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_1dfcdbef-d660-4977-8363-0facffed5dd7
8652023-12-06T17:42:02.190ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_1dfcdbef-d660-4977-8363-0facffed5dd7") failed: Property not found. retry in 129.556149ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_1dfcdbef-d660-4977-8363-0facffed5dd7
8662023-12-06T17:42:02.300ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_51de6a17-3c29-4573-9303-268fbfac8ee2") failed: Property not found. retry in 491.692692ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_51de6a17-3c29-4573-9303-268fbfac8ee2
8672023-12-06T17:42:02.364ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_1dfcdbef-d660-4977-8363-0facffed5dd7") failed: Property not found. retry in 583.028379ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_1dfcdbef-d660-4977-8363-0facffed5dd7
8682023-12-06T17:42:02.594ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_f4bd4239-a995-46ad-bd81-8c0c7f2f9049") failed: Property not found. retry in 1.011972093s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_f4bd4239-a995-46ad-bd81-8c0c7f2f9049
8692023-12-06T17:42:02.686ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_f6447f87-90dd-43a9-ac51-9e1831c10c3d") failed: Property not found. retry in 1.080290258s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_f6447f87-90dd-43a9-ac51-9e1831c10c3d
8702023-12-06T17:42:02.723ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_9bbb87d1-f50a-41e0-85ed-19784220ea16") failed: Property not found. retry in 1.324860811s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_9bbb87d1-f50a-41e0-85ed-19784220ea16
8712023-12-06T17:42:02.806ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_51de6a17-3c29-4573-9303-268fbfac8ee2") failed: Property not found. retry in 830.315641ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_51de6a17-3c29-4573-9303-268fbfac8ee2
8722023-12-06T17:42:02.836ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_993ea282-1eee-42e3-b3d8-ea8aa4c06162") failed: Property not found. retry in 1.076371294s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_993ea282-1eee-42e3-b3d8-ea8aa4c06162
8732023-12-06T17:42:02.913ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_pantry_0bfeea29-a47a-4323-893a-53a06601ad8e: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/crucible/pantry"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value="oxControlService13"/> <propval type="astring" name="gateway" value="fd00:1122:3344:101::1"/> <propval type="astring" name="listen_addr" value="fd00:1122:3344:101::11"/> <propval type="astring" name="listen_port" value="17000"/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
8742023-12-06T17:42:02.913ZINFOSledAgent (ServiceManager): Zone booting file = illumos-utils/src/running_zone.rs:512 zone = oxz_crucible_pantry_0bfeea29-a47a-4323-893a-53a06601ad8e
8752023-12-06T17:42:02.961ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_1dfcdbef-d660-4977-8363-0facffed5dd7") failed: Property not found. retry in 901.335591ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_1dfcdbef-d660-4977-8363-0facffed5dd7
8762023-12-06T17:42:03.138ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_65a5c94d-7dd3-419e-b0c4-23533a58135d: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/crucible/agent"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value="oxControlService11"/> <propval type="astring" name="gateway" value="fd00:1122:3344:101::1"/> <propval type="astring" name="dataset" value="oxp_e4b4dc87-ab46-49fb-a4b4-d361ae214c03/crucible"/> <propval type="astring" name="listen_addr" value="fd00:1122:3344:101::19"/> <propval type="astring" name="listen_port" value="32345"/> <propval type="astring" name="uuid" value="cdcb0243-1aeb-41b3-825c-42951073eb5c"/> <propval type="astring" name="store" value="/data"/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
8772023-12-06T17:42:03.139ZINFOSledAgent (ServiceManager): Zone booting file = illumos-utils/src/running_zone.rs:512 zone = oxz_crucible_65a5c94d-7dd3-419e-b0c4-23533a58135d
8782023-12-06T17:42:03.156ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_0bfeea29-a47a-4323-893a-53a06601ad8e") failed: Property not found. retry in 29.107618ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_0bfeea29-a47a-4323-893a-53a06601ad8e
8792023-12-06T17:42:03.170ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_f92ea1e7-347c-41d1-bd44-a15ef90d284c: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/crucible/agent"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value="oxControlService15"/> <propval type="astring" name="gateway" value="fd00:1122:3344:101::1"/> <propval type="astring" name="dataset" value="oxp_616b26df-e62a-4c68-b506-f4a923d8aaf7/crucible"/> <propval type="astring" name="listen_addr" value="fd00:1122:3344:101::15"/> <propval type="astring" name="listen_port" value="32345"/> <propval type="astring" name="uuid" value="b71b90e8-034a-47e2-a22c-327fa8e44bf6"/> <propval type="astring" name="store" value="/data"/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
8802023-12-06T17:42:03.170ZINFOSledAgent (ServiceManager): Zone booting file = illumos-utils/src/running_zone.rs:512 zone = oxz_crucible_f92ea1e7-347c-41d1-bd44-a15ef90d284c
8812023-12-06T17:42:03.199ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_0bfeea29-a47a-4323-893a-53a06601ad8e") failed: Property not found. retry in 85.465933ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_0bfeea29-a47a-4323-893a-53a06601ad8e
8822023-12-06T17:42:03.298ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_0bfeea29-a47a-4323-893a-53a06601ad8e") failed: Property not found. retry in 238.18996ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_0bfeea29-a47a-4323-893a-53a06601ad8e
8832023-12-06T17:42:03.341ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_pantry_5f32b190-8672-494f-90a3-7cd7719c5987: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/crucible/pantry"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value="oxControlService17"/> <propval type="astring" name="gateway" value="fd00:1122:3344:101::1"/> <propval type="astring" name="listen_addr" value="fd00:1122:3344:101::10"/> <propval type="astring" name="listen_port" value="17000"/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
8842023-12-06T17:42:03.342ZINFOSledAgent (ServiceManager): Zone booting file = illumos-utils/src/running_zone.rs:512 zone = oxz_crucible_pantry_5f32b190-8672-494f-90a3-7cd7719c5987
8852023-12-06T17:42:03.363ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_65a5c94d-7dd3-419e-b0c4-23533a58135d") failed: Property not found. retry in 26.494819ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_65a5c94d-7dd3-419e-b0c4-23533a58135d
8862023-12-06T17:42:03.402ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_65a5c94d-7dd3-419e-b0c4-23533a58135d") failed: Property not found. retry in 64.826614ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_65a5c94d-7dd3-419e-b0c4-23533a58135d
8872023-12-06T17:42:03.421ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_f92ea1e7-347c-41d1-bd44-a15ef90d284c") failed: Property not found. retry in 55.618433ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_f92ea1e7-347c-41d1-bd44-a15ef90d284c
8882023-12-06T17:42:03.457ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_f70a9b4a-82f3-4f7e-9b2e-db724f71ab50: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/crucible/agent"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value="oxControlService21"/> <propval type="astring" name="gateway" value="fd00:1122:3344:101::1"/> <propval type="astring" name="dataset" value="oxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crucible"/> <propval type="astring" name="listen_addr" value="fd00:1122:3344:101::1a"/> <propval type="astring" name="listen_port" value="32345"/> <propval type="astring" name="uuid" value="2a14cf6a-7fac-4c69-bf70-3932041d7c5d"/> <propval type="astring" name="store" value="/data"/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
8892023-12-06T17:42:03.457ZINFOSledAgent (ServiceManager): Zone booting file = illumos-utils/src/running_zone.rs:512 zone = oxz_crucible_f70a9b4a-82f3-4f7e-9b2e-db724f71ab50
8902023-12-06T17:42:03.478ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_65a5c94d-7dd3-419e-b0c4-23533a58135d") failed: Property not found. retry in 257.941899ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_65a5c94d-7dd3-419e-b0c4-23533a58135d
8912023-12-06T17:42:03.488ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_f92ea1e7-347c-41d1-bd44-a15ef90d284c") failed: Property not found. retry in 147.514761ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_f92ea1e7-347c-41d1-bd44-a15ef90d284c
8922023-12-06T17:42:03.495ZINFOSledAgent (ServiceManager): Zone booting file = illumos-utils/src/running_zone.rs:512 zone = oxz_oximeter_358b59f5-3c51-4c63-92f1-1dcada1dc798
8932023-12-06T17:42:03.525ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_873ec7ab-6024-4863-b381-a4fcad4788ec: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/crucible/agent"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value="oxControlService18"/> <propval type="astring" name="gateway" value="fd00:1122:3344:101::1"/> <propval type="astring" name="dataset" value="oxp_24b4dc87-ab46-49fb-a4b4-d361ae214c03/crucible"/> <propval type="astring" name="listen_addr" value="fd00:1122:3344:101::13"/> <propval type="astring" name="listen_port" value="32345"/> <propval type="astring" name="uuid" value="16f4364e-cf5a-453e-862f-79981ae74b6e"/> <propval type="astring" name="store" value="/data"/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
8942023-12-06T17:42:03.525ZINFOSledAgent (ServiceManager): Zone booting file = illumos-utils/src/running_zone.rs:512 zone = oxz_crucible_873ec7ab-6024-4863-b381-a4fcad4788ec
8952023-12-06T17:42:03.551ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_0bfeea29-a47a-4323-893a-53a06601ad8e") failed: Property not found. retry in 419.741783ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_0bfeea29-a47a-4323-893a-53a06601ad8e
8962023-12-06T17:42:03.573ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_5f32b190-8672-494f-90a3-7cd7719c5987") failed: Property not found. retry in 66.395129ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_5f32b190-8672-494f-90a3-7cd7719c5987
8972023-12-06T17:42:03.591ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_f148960a-0913-4686-9975-ff2972d55de0: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/crucible/agent"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value="oxControlService12"/> <propval type="astring" name="gateway" value="fd00:1122:3344:101::1"/> <propval type="astring" name="dataset" value="oxp_ceb4461c-cf56-4719-ad3c-14430bfdfb60/crucible"/> <propval type="astring" name="listen_addr" value="fd00:1122:3344:101::17"/> <propval type="astring" name="listen_port" value="32345"/> <propval type="astring" name="uuid" value="10817c6c-89b6-48cc-b0ac-35a0301be481"/> <propval type="astring" name="store" value="/data"/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
8982023-12-06T17:42:03.591ZINFOSledAgent (ServiceManager): Zone booting file = illumos-utils/src/running_zone.rs:512 zone = oxz_crucible_f148960a-0913-4686-9975-ff2972d55de0
8992023-12-06T17:42:03.618ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_f4bd4239-a995-46ad-bd81-8c0c7f2f9049") failed: Property not found. retry in 1.161067011s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_f4bd4239-a995-46ad-bd81-8c0c7f2f9049
9002023-12-06T17:42:03.623ZINFOSledAgent (ServiceManager): Zone booting file = illumos-utils/src/running_zone.rs:512 zone = oxz_external_dns_2cfabf1e-5b74-40a8-8d00-5f984444bf51
9012023-12-06T17:42:03.632ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_879f5844-21f5-4128-accd-2686d45ad2ab: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/crucible/agent"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value="oxControlService19"/> <propval type="astring" name="gateway" value="fd00:1122:3344:101::1"/> <propval type="astring" name="dataset" value="oxp_cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crucible"/> <propval type="astring" name="listen_addr" value="fd00:1122:3344:101::16"/> <propval type="astring" name="listen_port" value="32345"/> <propval type="astring" name="uuid" value="49c5d64f-e412-490f-b3a6-ad2bc6bca55d"/> <propval type="astring" name="store" value="/data"/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
9022023-12-06T17:42:03.632ZINFOSledAgent (ServiceManager): Zone booting file = illumos-utils/src/running_zone.rs:512 zone = oxz_crucible_879f5844-21f5-4128-accd-2686d45ad2ab
9032023-12-06T17:42:03.651ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_51de6a17-3c29-4573-9303-268fbfac8ee2") failed: Property not found. retry in 1.142232692s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_51de6a17-3c29-4573-9303-268fbfac8ee2
9042023-12-06T17:42:03.669ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_f92ea1e7-347c-41d1-bd44-a15ef90d284c") failed: Property not found. retry in 242.563535ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_f92ea1e7-347c-41d1-bd44-a15ef90d284c
9052023-12-06T17:42:03.683ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_5f32b190-8672-494f-90a3-7cd7719c5987") failed: Property not found. retry in 100.806463ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_5f32b190-8672-494f-90a3-7cd7719c5987
9062023-12-06T17:42:03.698ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_f70a9b4a-82f3-4f7e-9b2e-db724f71ab50") failed: Property not found. retry in 49.040649ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_f70a9b4a-82f3-4f7e-9b2e-db724f71ab50
9072023-12-06T17:42:03.761ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_65a5c94d-7dd3-419e-b0c4-23533a58135d") failed: Property not found. retry in 507.662039ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_65a5c94d-7dd3-419e-b0c4-23533a58135d
9082023-12-06T17:42:03.875ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_f70a9b4a-82f3-4f7e-9b2e-db724f71ab50") failed: Property not found. retry in 119.761166ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_f70a9b4a-82f3-4f7e-9b2e-db724f71ab50
9092023-12-06T17:42:03.925ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_358b59f5-3c51-4c63-92f1-1dcada1dc798") failed: Property not found. retry in 47.852971ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_358b59f5-3c51-4c63-92f1-1dcada1dc798
9102023-12-06T17:42:03.945ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_5f32b190-8672-494f-90a3-7cd7719c5987") failed: Property not found. retry in 194.536856ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_5f32b190-8672-494f-90a3-7cd7719c5987
9112023-12-06T17:42:03.969ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_f6447f87-90dd-43a9-ac51-9e1831c10c3d") failed: Property not found. retry in 1.26172724s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_f6447f87-90dd-43a9-ac51-9e1831c10c3d
9122023-12-06T17:42:04.014ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_1dfcdbef-d660-4977-8363-0facffed5dd7") failed: Property not found. retry in 952.980249ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_1dfcdbef-d660-4977-8363-0facffed5dd7
9132023-12-06T17:42:04.039ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_993ea282-1eee-42e3-b3d8-ea8aa4c06162") failed: Property not found. retry in 501.00817ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_993ea282-1eee-42e3-b3d8-ea8aa4c06162
9142023-12-06T17:42:04.051ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_f148960a-0913-4686-9975-ff2972d55de0") failed: Property not found. retry in 27.24968ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_f148960a-0913-4686-9975-ff2972d55de0
9152023-12-06T17:42:04.063ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_0bfeea29-a47a-4323-893a-53a06601ad8e") failed: Property not found. retry in 804.484911ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_0bfeea29-a47a-4323-893a-53a06601ad8e
9162023-12-06T17:42:04.078ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_9bbb87d1-f50a-41e0-85ed-19784220ea16") failed: Property not found. retry in 548.085225ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_9bbb87d1-f50a-41e0-85ed-19784220ea16
9172023-12-06T17:42:04.092ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_f92ea1e7-347c-41d1-bd44-a15ef90d284c") failed: Property not found. retry in 364.758214ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_f92ea1e7-347c-41d1-bd44-a15ef90d284c
9182023-12-06T17:42:04.092ZINFOSledAgent (ServiceManager): Zone booting file = illumos-utils/src/running_zone.rs:512 zone = oxz_nexus_d2016ddd-0efc-4439-89aa-2f166d289e6f
9192023-12-06T17:42:04.113ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_358b59f5-3c51-4c63-92f1-1dcada1dc798") failed: Property not found. retry in 67.020186ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_358b59f5-3c51-4c63-92f1-1dcada1dc798
9202023-12-06T17:42:04.125ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_873ec7ab-6024-4863-b381-a4fcad4788ec") failed: Property not found. retry in 43.5452ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_873ec7ab-6024-4863-b381-a4fcad4788ec
9212023-12-06T17:42:04.141ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_f70a9b4a-82f3-4f7e-9b2e-db724f71ab50") failed: Property not found. retry in 209.604554ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_f70a9b4a-82f3-4f7e-9b2e-db724f71ab50
9222023-12-06T17:42:04.153ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_f148960a-0913-4686-9975-ff2972d55de0") failed: Property not found. retry in 146.858536ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_f148960a-0913-4686-9975-ff2972d55de0
9232023-12-06T17:42:04.165ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_5f32b190-8672-494f-90a3-7cd7719c5987") failed: Property not found. retry in 407.947063ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_5f32b190-8672-494f-90a3-7cd7719c5987
9242023-12-06T17:42:04.175ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_879f5844-21f5-4128-accd-2686d45ad2ab") failed: Property not found. retry in 55.018173ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_879f5844-21f5-4128-accd-2686d45ad2ab
9252023-12-06T17:42:04.189ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_2cfabf1e-5b74-40a8-8d00-5f984444bf51") failed: Property not found. retry in 72.395254ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_2cfabf1e-5b74-40a8-8d00-5f984444bf51
9262023-12-06T17:42:04.205ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_873ec7ab-6024-4863-b381-a4fcad4788ec") failed: Property not found. retry in 139.019901ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_873ec7ab-6024-4863-b381-a4fcad4788ec
9272023-12-06T17:42:04.336ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::60fc:8ff:fe95:bcc8": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
9282023-12-06T17:42:04.352ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_358b59f5-3c51-4c63-92f1-1dcada1dc798") failed: Property not found. retry in 287.799765ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_358b59f5-3c51-4c63-92f1-1dcada1dc798
9292023-12-06T17:42:04.366ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_2cfabf1e-5b74-40a8-8d00-5f984444bf51") failed: Property not found. retry in 91.54867ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_2cfabf1e-5b74-40a8-8d00-5f984444bf51
9302023-12-06T17:42:04.381ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_873ec7ab-6024-4863-b381-a4fcad4788ec") failed: Property not found. retry in 178.791688ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_873ec7ab-6024-4863-b381-a4fcad4788ec
9312023-12-06T17:42:04.393ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_879f5844-21f5-4128-accd-2686d45ad2ab") failed: Property not found. retry in 65.656981ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_879f5844-21f5-4128-accd-2686d45ad2ab
9322023-12-06T17:42:04.408ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_f70a9b4a-82f3-4f7e-9b2e-db724f71ab50") failed: Property not found. retry in 249.837413ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_f70a9b4a-82f3-4f7e-9b2e-db724f71ab50
9332023-12-06T17:42:04.420ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_65a5c94d-7dd3-419e-b0c4-23533a58135d") failed: Property not found. retry in 1.088683403s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_65a5c94d-7dd3-419e-b0c4-23533a58135d
9342023-12-06T17:42:04.435ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_f148960a-0913-4686-9975-ff2972d55de0") failed: Property not found. retry in 189.457774ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_f148960a-0913-4686-9975-ff2972d55de0
9352023-12-06T17:42:04.436ZINFOSledAgent (ServiceManager): Zone booting file = illumos-utils/src/running_zone.rs:512 zone = oxz_nexus_9a25727f-e844-4dd1-b1b4-978eca7f5c90
9362023-12-06T17:42:04.471ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_f92ea1e7-347c-41d1-bd44-a15ef90d284c") failed: Property not found. retry in 1.034872944s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_f92ea1e7-347c-41d1-bd44-a15ef90d284c
9372023-12-06T17:42:04.482ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_2cfabf1e-5b74-40a8-8d00-5f984444bf51") failed: Property not found. retry in 212.574606ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_2cfabf1e-5b74-40a8-8d00-5f984444bf51
9382023-12-06T17:42:04.495ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_879f5844-21f5-4128-accd-2686d45ad2ab") failed: Property not found. retry in 210.105637ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_879f5844-21f5-4128-accd-2686d45ad2ab
9392023-12-06T17:42:04.505ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_d2016ddd-0efc-4439-89aa-2f166d289e6f") failed: Property not found. retry in 45.873677ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_d2016ddd-0efc-4439-89aa-2f166d289e6f
9402023-12-06T17:42:04.555ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_993ea282-1eee-42e3-b3d8-ea8aa4c06162") failed: Property not found. retry in 1.351181096s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_993ea282-1eee-42e3-b3d8-ea8aa4c06162
9412023-12-06T17:42:04.568ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_d2016ddd-0efc-4439-89aa-2f166d289e6f") failed: Property not found. retry in 146.689885ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_d2016ddd-0efc-4439-89aa-2f166d289e6f
9422023-12-06T17:42:04.581ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_873ec7ab-6024-4863-b381-a4fcad4788ec") failed: Property not found. retry in 434.154417ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_873ec7ab-6024-4863-b381-a4fcad4788ec
9432023-12-06T17:42:04.596ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_5f32b190-8672-494f-90a3-7cd7719c5987") failed: Property not found. retry in 510.236102ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_5f32b190-8672-494f-90a3-7cd7719c5987
9442023-12-06T17:42:04.643ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_f148960a-0913-4686-9975-ff2972d55de0") failed: Property not found. retry in 438.709185ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_f148960a-0913-4686-9975-ff2972d55de0
9452023-12-06T17:42:04.656ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_9bbb87d1-f50a-41e0-85ed-19784220ea16") failed: Property not found. retry in 875.976261ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_9bbb87d1-f50a-41e0-85ed-19784220ea16
9462023-12-06T17:42:04.670ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_358b59f5-3c51-4c63-92f1-1dcada1dc798") failed: Property not found. retry in 528.660338ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_358b59f5-3c51-4c63-92f1-1dcada1dc798
9472023-12-06T17:42:04.683ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_f70a9b4a-82f3-4f7e-9b2e-db724f71ab50") failed: Property not found. retry in 746.305346ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_f70a9b4a-82f3-4f7e-9b2e-db724f71ab50
9482023-12-06T17:42:04.694ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_9a25727f-e844-4dd1-b1b4-978eca7f5c90") failed: Property not found. retry in 70.143158ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_9a25727f-e844-4dd1-b1b4-978eca7f5c90
9492023-12-06T17:42:04.708ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_2cfabf1e-5b74-40a8-8d00-5f984444bf51") failed: Property not found. retry in 224.533843ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_2cfabf1e-5b74-40a8-8d00-5f984444bf51
9502023-12-06T17:42:04.718ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_879f5844-21f5-4128-accd-2686d45ad2ab") failed: Property not found. retry in 342.48682ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_879f5844-21f5-4128-accd-2686d45ad2ab
9512023-12-06T17:42:04.729ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_d2016ddd-0efc-4439-89aa-2f166d289e6f") failed: Property not found. retry in 153.42102ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_d2016ddd-0efc-4439-89aa-2f166d289e6f
9522023-12-06T17:42:04.778ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_9a25727f-e844-4dd1-b1b4-978eca7f5c90") failed: Property not found. retry in 90.077523ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_9a25727f-e844-4dd1-b1b4-978eca7f5c90
9532023-12-06T17:42:04.793ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_f4bd4239-a995-46ad-bd81-8c0c7f2f9049") failed: Property not found. retry in 579.948815ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_f4bd4239-a995-46ad-bd81-8c0c7f2f9049
9542023-12-06T17:42:04.807ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_51de6a17-3c29-4573-9303-268fbfac8ee2") failed: Property not found. retry in 849.428745ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_51de6a17-3c29-4573-9303-268fbfac8ee2
9552023-12-06T17:42:04.882ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_9a25727f-e844-4dd1-b1b4-978eca7f5c90") failed: Property not found. retry in 234.169751ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_9a25727f-e844-4dd1-b1b4-978eca7f5c90
9562023-12-06T17:42:04.895ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_0bfeea29-a47a-4323-893a-53a06601ad8e") failed: Property not found. retry in 603.786666ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_0bfeea29-a47a-4323-893a-53a06601ad8e
9572023-12-06T17:42:04.911ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_d2016ddd-0efc-4439-89aa-2f166d289e6f") failed: Property not found. retry in 470.66377ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_d2016ddd-0efc-4439-89aa-2f166d289e6f
9582023-12-06T17:42:04.954ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_2cfabf1e-5b74-40a8-8d00-5f984444bf51") failed: Property not found. retry in 474.039755ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_2cfabf1e-5b74-40a8-8d00-5f984444bf51
9592023-12-06T17:42:05.042ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_1dfcdbef-d660-4977-8363-0facffed5dd7") failed: Property not found. retry in 914.322532ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_1dfcdbef-d660-4977-8363-0facffed5dd7
9602023-12-06T17:42:05.078ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_873ec7ab-6024-4863-b381-a4fcad4788ec") failed: Property not found. retry in 1.157574188s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_873ec7ab-6024-4863-b381-a4fcad4788ec
9612023-12-06T17:42:05.127ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_879f5844-21f5-4128-accd-2686d45ad2ab") failed: Property not found. retry in 585.370756ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_879f5844-21f5-4128-accd-2686d45ad2ab
9622023-12-06T17:42:05.152ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_f148960a-0913-4686-9975-ff2972d55de0") failed: Property not found. retry in 578.380242ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_f148960a-0913-4686-9975-ff2972d55de0
9632023-12-06T17:42:05.184ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_9a25727f-e844-4dd1-b1b4-978eca7f5c90") failed: Property not found. retry in 575.209548ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_9a25727f-e844-4dd1-b1b4-978eca7f5c90
9642023-12-06T17:42:05.199ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_5f32b190-8672-494f-90a3-7cd7719c5987") failed: Property not found. retry in 959.063109ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_5f32b190-8672-494f-90a3-7cd7719c5987
9652023-12-06T17:42:05.215ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_358b59f5-3c51-4c63-92f1-1dcada1dc798") failed: Property not found. retry in 1.003146709s file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_358b59f5-3c51-4c63-92f1-1dcada1dc798
9662023-12-06T17:42:05.248ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_f6447f87-90dd-43a9-ac51-9e1831c10c3d") failed: Property not found. retry in 521.364061ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_f6447f87-90dd-43a9-ac51-9e1831c10c3d
9672023-12-06T17:42:05.386ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_f4bd4239-a995-46ad-bd81-8c0c7f2f9049") failed: Property not found. retry in 679.931823ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_f4bd4239-a995-46ad-bd81-8c0c7f2f9049
9682023-12-06T17:42:05.399ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_d2016ddd-0efc-4439-89aa-2f166d289e6f") failed: Property not found. retry in 826.373713ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_d2016ddd-0efc-4439-89aa-2f166d289e6f
9692023-12-06T17:42:05.444ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_2cfabf1e-5b74-40a8-8d00-5f984444bf51") failed: Property not found. retry in 1.23339021s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_2cfabf1e-5b74-40a8-8d00-5f984444bf51
9702023-12-06T17:42:05.458ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_f70a9b4a-82f3-4f7e-9b2e-db724f71ab50") failed: Property not found. retry in 616.605204ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_f70a9b4a-82f3-4f7e-9b2e-db724f71ab50
9712023-12-06T17:42:05.518ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_0bfeea29-a47a-4323-893a-53a06601ad8e") failed: Property not found. retry in 527.091833ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_0bfeea29-a47a-4323-893a-53a06601ad8e
9722023-12-06T17:42:05.534ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_65a5c94d-7dd3-419e-b0c4-23533a58135d") failed: Property not found. retry in 815.307604ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_65a5c94d-7dd3-419e-b0c4-23533a58135d
9732023-12-06T17:42:05.548ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_f92ea1e7-347c-41d1-bd44-a15ef90d284c") failed: Property not found. retry in 924.925062ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_f92ea1e7-347c-41d1-bd44-a15ef90d284c
9742023-12-06T17:42:05.568ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_9bbb87d1-f50a-41e0-85ed-19784220ea16") failed: Property not found. retry in 537.673167ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_9bbb87d1-f50a-41e0-85ed-19784220ea16
9752023-12-06T17:42:05.678ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_51de6a17-3c29-4573-9303-268fbfac8ee2") failed: Property not found. retry in 863.942883ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_51de6a17-3c29-4573-9303-268fbfac8ee2
9762023-12-06T17:42:05.728ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_879f5844-21f5-4128-accd-2686d45ad2ab") failed: Property not found. retry in 1.123570834s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_879f5844-21f5-4128-accd-2686d45ad2ab
9772023-12-06T17:42:05.750ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_f148960a-0913-4686-9975-ff2972d55de0") failed: Property not found. retry in 769.818879ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_f148960a-0913-4686-9975-ff2972d55de0
9782023-12-06T17:42:05.777ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_9a25727f-e844-4dd1-b1b4-978eca7f5c90") failed: Property not found. retry in 994.453747ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_9a25727f-e844-4dd1-b1b4-978eca7f5c90
9792023-12-06T17:42:05.796ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_f6447f87-90dd-43a9-ac51-9e1831c10c3d") failed: Property not found. retry in 655.712043ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_f6447f87-90dd-43a9-ac51-9e1831c10c3d
9802023-12-06T17:42:05.923ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_993ea282-1eee-42e3-b3d8-ea8aa4c06162") failed: Property not found. retry in 1.369130765s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_993ea282-1eee-42e3-b3d8-ea8aa4c06162
9812023-12-06T17:42:05.979ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_1dfcdbef-d660-4977-8363-0facffed5dd7") failed: Property not found. retry in 646.176207ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_1dfcdbef-d660-4977-8363-0facffed5dd7
9822023-12-06T17:42:05.982ZINFOSledAgent (ServiceManager): Profile for oxz_clickhouse_345211c8-e540-44f1-be03-4dfe81553f49: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/clickhouse"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value="oxControlService10"/> <propval type="astring" name="gateway" value="fd00:1122:3344:101::1"/> <propval type="astring" name="listen_addr" value="fd00:1122:3344:101::e"/> <propval type="astring" name="listen_port" value="8123"/> <propval type="astring" name="store" value="/data"/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/install"> <property_group type="application" name="install_props"> <property type="net_address" name="nameserver"> <net_address_list> <value_node value="fd00:1122:3344:1::1" /> <value_node value="fd00:1122:3344:3::1" /> <value_node value="fd00:1122:3344:2::1" /> </net_address_list> </property> </property_group> <instance enabled="true" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
9832023-12-06T17:42:05.983ZINFOSledAgent (ServiceManager): Zone booting file = illumos-utils/src/running_zone.rs:512 zone = oxz_clickhouse_345211c8-e540-44f1-be03-4dfe81553f49
9842023-12-06T17:42:06.062ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_0bfeea29-a47a-4323-893a-53a06601ad8e") failed: Property not found. retry in 543.390314ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_0bfeea29-a47a-4323-893a-53a06601ad8e
9852023-12-06T17:42:06.087ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_f4bd4239-a995-46ad-bd81-8c0c7f2f9049") failed: Property not found. retry in 816.69929ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_f4bd4239-a995-46ad-bd81-8c0c7f2f9049
9862023-12-06T17:42:06.107ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_f70a9b4a-82f3-4f7e-9b2e-db724f71ab50") failed: Property not found. retry in 670.442054ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_f70a9b4a-82f3-4f7e-9b2e-db724f71ab50
9872023-12-06T17:42:06.134ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_9bbb87d1-f50a-41e0-85ed-19784220ea16") failed: Property not found. retry in 1.3080495s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_9bbb87d1-f50a-41e0-85ed-19784220ea16
9882023-12-06T17:42:06.181ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_5f32b190-8672-494f-90a3-7cd7719c5987") failed: Property not found. retry in 997.888197ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_5f32b190-8672-494f-90a3-7cd7719c5987
9892023-12-06T17:42:06.239ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_358b59f5-3c51-4c63-92f1-1dcada1dc798") failed: Property not found. retry in 1.485960748s file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_358b59f5-3c51-4c63-92f1-1dcada1dc798
9902023-12-06T17:42:06.250ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:769 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:61892
9912023-12-06T17:42:06.250ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 139 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:61892 req_id = 692f3785-8bf7-4d40-ba5f-cc7e4cdfa349 response_code = 200 uri = /baseboard
9922023-12-06T17:42:06.257ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_d2016ddd-0efc-4439-89aa-2f166d289e6f") failed: Property not found. retry in 948.994033ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_d2016ddd-0efc-4439-89aa-2f166d289e6f
9932023-12-06T17:42:06.275ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_873ec7ab-6024-4863-b381-a4fcad4788ec") failed: Property not found. retry in 1.376534764s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_873ec7ab-6024-4863-b381-a4fcad4788ec
9942023-12-06T17:42:06.306ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_345211c8-e540-44f1-be03-4dfe81553f49") failed: Property not found. retry in 66.007436ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_345211c8-e540-44f1-be03-4dfe81553f49
9952023-12-06T17:42:06.372ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_65a5c94d-7dd3-419e-b0c4-23533a58135d") failed: Property not found. retry in 1.37484088s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_65a5c94d-7dd3-419e-b0c4-23533a58135d
9962023-12-06T17:42:06.389ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_345211c8-e540-44f1-be03-4dfe81553f49") failed: Property not found. retry in 58.214633ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_345211c8-e540-44f1-be03-4dfe81553f49
9972023-12-06T17:42:06.484ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_f6447f87-90dd-43a9-ac51-9e1831c10c3d") failed: Property not found. retry in 960.635272ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_f6447f87-90dd-43a9-ac51-9e1831c10c3d
9982023-12-06T17:42:06.522ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_345211c8-e540-44f1-be03-4dfe81553f49") failed: Property not found. retry in 126.650264ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_345211c8-e540-44f1-be03-4dfe81553f49
9992023-12-06T17:42:06.576ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_f92ea1e7-347c-41d1-bd44-a15ef90d284c") failed: Property not found. retry in 1.347838256s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_f92ea1e7-347c-41d1-bd44-a15ef90d284c
10002023-12-06T17:42:06.592ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_f148960a-0913-4686-9975-ff2972d55de0") failed: Property not found. retry in 953.821818ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_f148960a-0913-4686-9975-ff2972d55de0
10012023-12-06T17:42:06.612ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_51de6a17-3c29-4573-9303-268fbfac8ee2") failed: Property not found. retry in 1.494208592s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_51de6a17-3c29-4573-9303-268fbfac8ee2
10022023-12-06T17:42:06.627ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_0bfeea29-a47a-4323-893a-53a06601ad8e") failed: Property not found. retry in 1.397513346s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_0bfeea29-a47a-4323-893a-53a06601ad8e
10032023-12-06T17:42:06.645ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_1dfcdbef-d660-4977-8363-0facffed5dd7") failed: Property not found. retry in 526.819863ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_1dfcdbef-d660-4977-8363-0facffed5dd7
10042023-12-06T17:42:06.667ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_345211c8-e540-44f1-be03-4dfe81553f49") failed: Property not found. retry in 347.898465ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_345211c8-e540-44f1-be03-4dfe81553f49
10052023-12-06T17:42:06.694ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_2cfabf1e-5b74-40a8-8d00-5f984444bf51") failed: Property not found. retry in 984.44506ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_2cfabf1e-5b74-40a8-8d00-5f984444bf51
10062023-12-06T17:42:06.796ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_9a25727f-e844-4dd1-b1b4-978eca7f5c90") failed: Property not found. retry in 739.017672ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_9a25727f-e844-4dd1-b1b4-978eca7f5c90
10072023-12-06T17:42:06.821ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_f70a9b4a-82f3-4f7e-9b2e-db724f71ab50") failed: Property not found. retry in 1.435629307s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_f70a9b4a-82f3-4f7e-9b2e-db724f71ab50
10082023-12-06T17:42:06.878ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_879f5844-21f5-4128-accd-2686d45ad2ab") failed: Property not found. retry in 1.234418248s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_879f5844-21f5-4128-accd-2686d45ad2ab
10092023-12-06T17:42:06.944ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_f4bd4239-a995-46ad-bd81-8c0c7f2f9049") failed: Property not found. retry in 1.098283739s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_f4bd4239-a995-46ad-bd81-8c0c7f2f9049
10102023-12-06T17:42:07.046ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_345211c8-e540-44f1-be03-4dfe81553f49") failed: Property not found. retry in 424.324347ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_345211c8-e540-44f1-be03-4dfe81553f49
10112023-12-06T17:42:07.192ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_1dfcdbef-d660-4977-8363-0facffed5dd7") failed: Property not found. retry in 1.315966503s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_1dfcdbef-d660-4977-8363-0facffed5dd7
10122023-12-06T17:42:07.210ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_5f32b190-8672-494f-90a3-7cd7719c5987") failed: Property not found. retry in 994.53924ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_5f32b190-8672-494f-90a3-7cd7719c5987
10132023-12-06T17:42:07.232ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_d2016ddd-0efc-4439-89aa-2f166d289e6f") failed: Property not found. retry in 1.1638422s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_d2016ddd-0efc-4439-89aa-2f166d289e6f
10142023-12-06T17:42:07.313ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_993ea282-1eee-42e3-b3d8-ea8aa4c06162") failed: Property not found. retry in 573.357162ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_993ea282-1eee-42e3-b3d8-ea8aa4c06162
10152023-12-06T17:42:07.478ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_9bbb87d1-f50a-41e0-85ed-19784220ea16") failed: Property not found. retry in 1.117931509s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_9bbb87d1-f50a-41e0-85ed-19784220ea16
10162023-12-06T17:42:07.550ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_345211c8-e540-44f1-be03-4dfe81553f49") failed: Property not found. retry in 1.440861566s file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_345211c8-e540-44f1-be03-4dfe81553f49
10172023-12-06T17:42:07.588ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_f6447f87-90dd-43a9-ac51-9e1831c10c3d") failed: Property not found. retry in 568.302745ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_f6447f87-90dd-43a9-ac51-9e1831c10c3d
10182023-12-06T17:42:07.611ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_f148960a-0913-4686-9975-ff2972d55de0") failed: Property not found. retry in 698.820315ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_f148960a-0913-4686-9975-ff2972d55de0
10192023-12-06T17:42:07.636ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_9a25727f-e844-4dd1-b1b4-978eca7f5c90") failed: Property not found. retry in 868.681558ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_9a25727f-e844-4dd1-b1b4-978eca7f5c90
10202023-12-06T17:42:07.674ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_873ec7ab-6024-4863-b381-a4fcad4788ec") failed: Property not found. retry in 795.768409ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_873ec7ab-6024-4863-b381-a4fcad4788ec
10212023-12-06T17:42:07.701ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_2cfabf1e-5b74-40a8-8d00-5f984444bf51") failed: Property not found. retry in 772.831713ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_2cfabf1e-5b74-40a8-8d00-5f984444bf51
10222023-12-06T17:42:07.744ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_358b59f5-3c51-4c63-92f1-1dcada1dc798") failed: Property not found. retry in 871.325646ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_358b59f5-3c51-4c63-92f1-1dcada1dc798
10232023-12-06T17:42:07.765ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_65a5c94d-7dd3-419e-b0c4-23533a58135d") failed: Property not found. retry in 1.037166886s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_65a5c94d-7dd3-419e-b0c4-23533a58135d
10242023-12-06T17:42:07.904ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_993ea282-1eee-42e3-b3d8-ea8aa4c06162") failed: Property not found. retry in 898.057999ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_993ea282-1eee-42e3-b3d8-ea8aa4c06162
10252023-12-06T17:42:07.944ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_f92ea1e7-347c-41d1-bd44-a15ef90d284c") failed: Property not found. retry in 1.330911531s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_f92ea1e7-347c-41d1-bd44-a15ef90d284c
10262023-12-06T17:42:08.042ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_0bfeea29-a47a-4323-893a-53a06601ad8e") failed: Property not found. retry in 986.367151ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_0bfeea29-a47a-4323-893a-53a06601ad8e
10272023-12-06T17:42:08.062ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_f4bd4239-a995-46ad-bd81-8c0c7f2f9049") failed: Property not found. retry in 1.001335872s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_f4bd4239-a995-46ad-bd81-8c0c7f2f9049
10282023-12-06T17:42:08.124ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_51de6a17-3c29-4573-9303-268fbfac8ee2") failed: Property not found. retry in 796.6017ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_51de6a17-3c29-4573-9303-268fbfac8ee2
10292023-12-06T17:42:08.140ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_879f5844-21f5-4128-accd-2686d45ad2ab") failed: Property not found. retry in 972.867032ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_879f5844-21f5-4128-accd-2686d45ad2ab
10302023-12-06T17:42:08.173ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_f6447f87-90dd-43a9-ac51-9e1831c10c3d") failed: Property not found. retry in 612.133038ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_f6447f87-90dd-43a9-ac51-9e1831c10c3d
10312023-12-06T17:42:08.220ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_5f32b190-8672-494f-90a3-7cd7719c5987") failed: Property not found. retry in 1.244311418s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_5f32b190-8672-494f-90a3-7cd7719c5987
10322023-12-06T17:42:08.270ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_f70a9b4a-82f3-4f7e-9b2e-db724f71ab50") failed: Property not found. retry in 913.950242ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_f70a9b4a-82f3-4f7e-9b2e-db724f71ab50
10332023-12-06T17:42:08.326ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_f148960a-0913-4686-9975-ff2972d55de0") failed: Property not found. retry in 776.02964ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_f148960a-0913-4686-9975-ff2972d55de0
10342023-12-06T17:42:08.411ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_d2016ddd-0efc-4439-89aa-2f166d289e6f") failed: Property not found. retry in 1.163190199s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_d2016ddd-0efc-4439-89aa-2f166d289e6f
10352023-12-06T17:42:08.484ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_873ec7ab-6024-4863-b381-a4fcad4788ec") failed: Property not found. retry in 1.401895794s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_873ec7ab-6024-4863-b381-a4fcad4788ec
10362023-12-06T17:42:08.499ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_2cfabf1e-5b74-40a8-8d00-5f984444bf51") failed: Property not found. retry in 858.096034ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_2cfabf1e-5b74-40a8-8d00-5f984444bf51
10372023-12-06T17:42:08.519ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_9a25727f-e844-4dd1-b1b4-978eca7f5c90") failed: Property not found. retry in 1.05447526s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_9a25727f-e844-4dd1-b1b4-978eca7f5c90
10382023-12-06T17:42:08.534ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_1dfcdbef-d660-4977-8363-0facffed5dd7") failed: Property not found. retry in 1.372601693s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_1dfcdbef-d660-4977-8363-0facffed5dd7
10392023-12-06T17:42:08.609ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_9bbb87d1-f50a-41e0-85ed-19784220ea16") failed: Property not found. retry in 1.139075374s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_9bbb87d1-f50a-41e0-85ed-19784220ea16
10402023-12-06T17:42:08.630ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_358b59f5-3c51-4c63-92f1-1dcada1dc798") failed: Property not found. retry in 1.317819921s file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_358b59f5-3c51-4c63-92f1-1dcada1dc798
10412023-12-06T17:42:08.798ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_f6447f87-90dd-43a9-ac51-9e1831c10c3d") failed: Property not found. retry in 1.111282587s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_f6447f87-90dd-43a9-ac51-9e1831c10c3d
10422023-12-06T17:42:08.816ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_993ea282-1eee-42e3-b3d8-ea8aa4c06162") failed: Property not found. retry in 556.957543ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_993ea282-1eee-42e3-b3d8-ea8aa4c06162
10432023-12-06T17:42:08.828ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_65a5c94d-7dd3-419e-b0c4-23533a58135d") failed: Property not found. retry in 1.320771043s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_65a5c94d-7dd3-419e-b0c4-23533a58135d
10442023-12-06T17:42:08.934ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_51de6a17-3c29-4573-9303-268fbfac8ee2") failed: Property not found. retry in 978.907719ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_51de6a17-3c29-4573-9303-268fbfac8ee2
10452023-12-06T17:42:09.002ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_345211c8-e540-44f1-be03-4dfe81553f49") failed: Property not found. retry in 1.220401299s file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_345211c8-e540-44f1-be03-4dfe81553f49
10462023-12-06T17:42:09.042ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_0bfeea29-a47a-4323-893a-53a06601ad8e") failed: Property not found. retry in 1.380118725s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_0bfeea29-a47a-4323-893a-53a06601ad8e
10472023-12-06T17:42:09.075ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_f4bd4239-a995-46ad-bd81-8c0c7f2f9049") failed: Property not found. retry in 824.87903ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_f4bd4239-a995-46ad-bd81-8c0c7f2f9049
10482023-12-06T17:42:09.113ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_f148960a-0913-4686-9975-ff2972d55de0") failed: Property not found. retry in 1.003271563s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_f148960a-0913-4686-9975-ff2972d55de0
10492023-12-06T17:42:09.122ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_879f5844-21f5-4128-accd-2686d45ad2ab") failed: Property not found. retry in 739.978104ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_879f5844-21f5-4128-accd-2686d45ad2ab
10502023-12-06T17:42:09.197ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_f70a9b4a-82f3-4f7e-9b2e-db724f71ab50") failed: Property not found. retry in 1.33305219s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_f70a9b4a-82f3-4f7e-9b2e-db724f71ab50
10512023-12-06T17:42:09.290ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_f92ea1e7-347c-41d1-bd44-a15ef90d284c") failed: Property not found. retry in 664.566083ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_f92ea1e7-347c-41d1-bd44-a15ef90d284c
10522023-12-06T17:42:09.341ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::60fc:8ff:fe95:bcc8": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
10532023-12-06T17:42:09.371ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_2cfabf1e-5b74-40a8-8d00-5f984444bf51") failed: Property not found. retry in 785.933644ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_2cfabf1e-5b74-40a8-8d00-5f984444bf51
10542023-12-06T17:42:09.385ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_993ea282-1eee-42e3-b3d8-ea8aa4c06162") failed: Property not found. retry in 557.797159ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_993ea282-1eee-42e3-b3d8-ea8aa4c06162
10552023-12-06T17:42:09.478ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_5f32b190-8672-494f-90a3-7cd7719c5987") failed: Property not found. retry in 1.147149487s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_5f32b190-8672-494f-90a3-7cd7719c5987
10562023-12-06T17:42:09.589ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_9a25727f-e844-4dd1-b1b4-978eca7f5c90") failed: Property not found. retry in 844.568661ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_9a25727f-e844-4dd1-b1b4-978eca7f5c90
10572023-12-06T17:42:09.601ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_d2016ddd-0efc-4439-89aa-2f166d289e6f") failed: Property not found. retry in 1.344575202s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_d2016ddd-0efc-4439-89aa-2f166d289e6f
10582023-12-06T17:42:09.760ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_9bbb87d1-f50a-41e0-85ed-19784220ea16") failed: Property not found. retry in 855.497256ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_9bbb87d1-f50a-41e0-85ed-19784220ea16
10592023-12-06T17:42:09.875ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_879f5844-21f5-4128-accd-2686d45ad2ab") failed: Property not found. retry in 824.12033ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_879f5844-21f5-4128-accd-2686d45ad2ab
10602023-12-06T17:42:09.897ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_873ec7ab-6024-4863-b381-a4fcad4788ec") failed: Property not found. retry in 677.035062ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_873ec7ab-6024-4863-b381-a4fcad4788ec
10612023-12-06T17:42:09.911ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_f4bd4239-a995-46ad-bd81-8c0c7f2f9049") failed: Property not found. retry in 1.241322338s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_f4bd4239-a995-46ad-bd81-8c0c7f2f9049
10622023-12-06T17:42:09.925ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_f6447f87-90dd-43a9-ac51-9e1831c10c3d") failed: Property not found. retry in 1.442655902s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_f6447f87-90dd-43a9-ac51-9e1831c10c3d
10632023-12-06T17:42:09.936ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_1dfcdbef-d660-4977-8363-0facffed5dd7") failed: Property not found. retry in 1.414675144s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_1dfcdbef-d660-4977-8363-0facffed5dd7
10642023-12-06T17:42:09.955ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_51de6a17-3c29-4573-9303-268fbfac8ee2") failed: Property not found. retry in 1.250749204s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_51de6a17-3c29-4573-9303-268fbfac8ee2
10652023-12-06T17:42:09.976ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_358b59f5-3c51-4c63-92f1-1dcada1dc798") failed: Property not found. retry in 519.538154ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_358b59f5-3c51-4c63-92f1-1dcada1dc798
10662023-12-06T17:42:09.990ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_993ea282-1eee-42e3-b3d8-ea8aa4c06162") failed: Property not found. retry in 738.283507ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_993ea282-1eee-42e3-b3d8-ea8aa4c06162
10672023-12-06T17:42:10.004ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_f92ea1e7-347c-41d1-bd44-a15ef90d284c") failed: Property not found. retry in 1.385149198s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_f92ea1e7-347c-41d1-bd44-a15ef90d284c
10682023-12-06T17:42:10.133ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_f148960a-0913-4686-9975-ff2972d55de0") failed: Property not found. retry in 910.309456ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_f148960a-0913-4686-9975-ff2972d55de0
10692023-12-06T17:42:10.164ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_65a5c94d-7dd3-419e-b0c4-23533a58135d") failed: Property not found. retry in 1.115302279s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_65a5c94d-7dd3-419e-b0c4-23533a58135d
10702023-12-06T17:42:10.177ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_2cfabf1e-5b74-40a8-8d00-5f984444bf51") failed: Property not found. retry in 1.480621593s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_2cfabf1e-5b74-40a8-8d00-5f984444bf51
10712023-12-06T17:42:10.241ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_345211c8-e540-44f1-be03-4dfe81553f49") failed: Property not found. retry in 1.276575925s file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_345211c8-e540-44f1-be03-4dfe81553f49
10722023-12-06T17:42:10.436ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_0bfeea29-a47a-4323-893a-53a06601ad8e") failed: Property not found. retry in 714.062875ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_0bfeea29-a47a-4323-893a-53a06601ad8e
10732023-12-06T17:42:10.449ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_9a25727f-e844-4dd1-b1b4-978eca7f5c90") failed: Property not found. retry in 893.245529ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_9a25727f-e844-4dd1-b1b4-978eca7f5c90
10742023-12-06T17:42:10.511ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_358b59f5-3c51-4c63-92f1-1dcada1dc798") failed: Property not found. retry in 1.277057974s file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_358b59f5-3c51-4c63-92f1-1dcada1dc798
10752023-12-06T17:42:10.550ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_f70a9b4a-82f3-4f7e-9b2e-db724f71ab50") failed: Property not found. retry in 971.93292ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_f70a9b4a-82f3-4f7e-9b2e-db724f71ab50
10762023-12-06T17:42:10.588ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_873ec7ab-6024-4863-b381-a4fcad4788ec") failed: Property not found. retry in 542.299046ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_873ec7ab-6024-4863-b381-a4fcad4788ec
10772023-12-06T17:42:10.633ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_9bbb87d1-f50a-41e0-85ed-19784220ea16") failed: Property not found. retry in 1.104378145s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_9bbb87d1-f50a-41e0-85ed-19784220ea16
10782023-12-06T17:42:10.646ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_5f32b190-8672-494f-90a3-7cd7719c5987") failed: Property not found. retry in 1.490166463s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_5f32b190-8672-494f-90a3-7cd7719c5987
10792023-12-06T17:42:10.714ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_879f5844-21f5-4128-accd-2686d45ad2ab") failed: Property not found. retry in 640.919517ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_879f5844-21f5-4128-accd-2686d45ad2ab
10802023-12-06T17:42:10.744ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_993ea282-1eee-42e3-b3d8-ea8aa4c06162") failed: Property not found. retry in 651.142582ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_993ea282-1eee-42e3-b3d8-ea8aa4c06162
10812023-12-06T17:42:10.961ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_d2016ddd-0efc-4439-89aa-2f166d289e6f") failed: Property not found. retry in 1.232306128s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_d2016ddd-0efc-4439-89aa-2f166d289e6f
10822023-12-06T17:42:11.065ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_f148960a-0913-4686-9975-ff2972d55de0") failed: Property not found. retry in 716.8699ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_f148960a-0913-4686-9975-ff2972d55de0
10832023-12-06T17:42:11.148ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_873ec7ab-6024-4863-b381-a4fcad4788ec") failed: Property not found. retry in 657.715633ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_873ec7ab-6024-4863-b381-a4fcad4788ec
10842023-12-06T17:42:11.171ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_0bfeea29-a47a-4323-893a-53a06601ad8e") failed: Property not found. retry in 1.176757137s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_0bfeea29-a47a-4323-893a-53a06601ad8e
10852023-12-06T17:42:11.222ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_51de6a17-3c29-4573-9303-268fbfac8ee2") failed: Property not found. retry in 992.098731ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_51de6a17-3c29-4573-9303-268fbfac8ee2
10862023-12-06T17:42:11.264ZINFOSledAgent (ServiceManager): Ensuring address fd00:1122:3344:101::9 exists file = sled-agent/src/services.rs:1769
10872023-12-06T17:42:11.264ZINFOSledAgent (ServiceManager): Adding address: Static(V6(Ipv6Network { addr: fd00:1122:3344:101::9, prefix: 64 })) file = illumos-utils/src/running_zone.rs:608 zone = oxz_external_dns_f4bd4239-a995-46ad-bd81-8c0c7f2f9049
10882023-12-06T17:42:11.918ZINFOSledAgent (ServiceManager): Ensuring address fd00:1122:3344:101::9 exists - OK file = sled-agent/src/services.rs:1777
10892023-12-06T17:42:11.919ZINFOSledAgent (ServiceManager): Zone using sled underlay as gateway file = sled-agent/src/services.rs:1796
10902023-12-06T17:42:13.250ZINFOSledAgent (ServiceManager): Setting up external-dns service file = sled-agent/src/services.rs:1953
10912023-12-06T17:42:13.250ZINFOSledAgent (ServiceManager): Ensuring address for OPTE port file = illumos-utils/src/running_zone.rs:652 zone = oxz_external_dns_f4bd4239-a995-46ad-bd81-8c0c7f2f9049
10922023-12-06T17:42:19.401ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::60fc:8ff:fe95:bcc8": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
10932023-12-06T17:42:19.668ZINFOSledAgent (ServiceManager): Ensuring address fd00:1122:3344:101::b exists file = sled-agent/src/services.rs:1769
10942023-12-06T17:42:19.668ZINFOSledAgent (ServiceManager): Adding address: Static(V6(Ipv6Network { addr: fd00:1122:3344:101::b, prefix: 64 })) file = illumos-utils/src/running_zone.rs:608 zone = oxz_nexus_9a25727f-e844-4dd1-b1b4-978eca7f5c90
10952023-12-06T17:42:20.231ZINFOSledAgent (ServiceManager): Ensuring address fd00:1122:3344:101::b exists - OK file = sled-agent/src/services.rs:1777
10962023-12-06T17:42:20.231ZINFOSledAgent (ServiceManager): Zone using sled underlay as gateway file = sled-agent/src/services.rs:1796
10972023-12-06T17:42:20.446ZINFOSledAgent (ServiceManager): Setting up Nexus service file = sled-agent/src/services.rs:1840
10982023-12-06T17:42:20.446ZINFOSledAgent (ServiceManager): Ensuring address for OPTE port file = illumos-utils/src/running_zone.rs:652 zone = oxz_nexus_9a25727f-e844-4dd1-b1b4-978eca7f5c90
10992023-12-06T17:42:24.402ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::60fc:8ff:fe95:bcc8": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
11002023-12-06T17:42:26.484ZINFOSledAgent (ServiceManager): Ensuring address fd00:1122:3344:101::a exists file = sled-agent/src/services.rs:1769
11012023-12-06T17:42:26.484ZINFOSledAgent (ServiceManager): Adding address: Static(V6(Ipv6Network { addr: fd00:1122:3344:101::a, prefix: 64 })) file = illumos-utils/src/running_zone.rs:608 zone = oxz_nexus_d2016ddd-0efc-4439-89aa-2f166d289e6f
11022023-12-06T17:42:27.036ZINFOSledAgent (ServiceManager): Ensuring address fd00:1122:3344:101::a exists - OK file = sled-agent/src/services.rs:1777
11032023-12-06T17:42:27.036ZINFOSledAgent (ServiceManager): Zone using sled underlay as gateway file = sled-agent/src/services.rs:1796
11042023-12-06T17:42:27.252ZINFOSledAgent (ServiceManager): Setting up Nexus service file = sled-agent/src/services.rs:1840
11052023-12-06T17:42:27.252ZINFOSledAgent (ServiceManager): Ensuring address for OPTE port file = illumos-utils/src/running_zone.rs:652 zone = oxz_nexus_d2016ddd-0efc-4439-89aa-2f166d289e6f
11062023-12-06T17:42:29.403ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::60fc:8ff:fe95:bcc8": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
11072023-12-06T17:42:32.648ZINFOSledAgent (ServiceManager): Ensuring address fd00:1122:3344:101::d exists file = sled-agent/src/services.rs:1769
11082023-12-06T17:42:32.648ZINFOSledAgent (ServiceManager): Adding address: Static(V6(Ipv6Network { addr: fd00:1122:3344:101::d, prefix: 64 })) file = illumos-utils/src/running_zone.rs:608 zone = oxz_oximeter_358b59f5-3c51-4c63-92f1-1dcada1dc798
11092023-12-06T17:42:33.207ZINFOSledAgent (ServiceManager): Ensuring address fd00:1122:3344:101::d exists - OK file = sled-agent/src/services.rs:1777
11102023-12-06T17:42:33.207ZINFOSledAgent (ServiceManager): Zone using sled underlay as gateway file = sled-agent/src/services.rs:1796
11112023-12-06T17:42:33.400ZINFOSledAgent (ServiceManager): Setting up oximeter service file = sled-agent/src/services.rs:2064
11122023-12-06T17:42:33.474ZINFOSledAgent (ServiceManager): Ensuring address fd00:1122:3344:101::8 exists file = sled-agent/src/services.rs:1769
11132023-12-06T17:42:33.474ZINFOSledAgent (ServiceManager): Adding address: Static(V6(Ipv6Network { addr: fd00:1122:3344:101::8, prefix: 64 })) file = illumos-utils/src/running_zone.rs:608 zone = oxz_external_dns_2cfabf1e-5b74-40a8-8d00-5f984444bf51
11142023-12-06T17:42:34.037ZINFOSledAgent (ServiceManager): Ensuring address fd00:1122:3344:101::8 exists - OK file = sled-agent/src/services.rs:1777
11152023-12-06T17:42:34.037ZINFOSledAgent (ServiceManager): Zone using sled underlay as gateway file = sled-agent/src/services.rs:1796
11162023-12-06T17:42:34.228ZINFOSledAgent (ServiceManager): Setting up external-dns service file = sled-agent/src/services.rs:1953
11172023-12-06T17:42:34.228ZINFOSledAgent (ServiceManager): Ensuring address for OPTE port file = illumos-utils/src/running_zone.rs:652 zone = oxz_external_dns_2cfabf1e-5b74-40a8-8d00-5f984444bf51
11182023-12-06T17:42:34.404ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::60fc:8ff:fe95:bcc8": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
11192023-12-06T17:42:36.286ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:769 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:58023
11202023-12-06T17:42:36.286ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 93 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:58023 req_id = 14d8516d-85cd-462b-9753-ea3474327fa4 response_code = 200 uri = /baseboard
11212023-12-06T17:42:39.406ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::60fc:8ff:fe95:bcc8": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
11222023-12-06T17:42:39.775ZINFOSledAgent (ServiceManager): Ensuring address fd00:1122:3344:101::c exists file = sled-agent/src/services.rs:1769
11232023-12-06T17:42:39.775ZINFOSledAgent (ServiceManager): Adding address: Static(V6(Ipv6Network { addr: fd00:1122:3344:101::c, prefix: 64 })) file = illumos-utils/src/running_zone.rs:608 zone = oxz_nexus_1dfcdbef-d660-4977-8363-0facffed5dd7
11242023-12-06T17:42:40.334ZINFOSledAgent (ServiceManager): Ensuring address fd00:1122:3344:101::c exists - OK file = sled-agent/src/services.rs:1777
11252023-12-06T17:42:40.335ZINFOSledAgent (ServiceManager): Zone using sled underlay as gateway file = sled-agent/src/services.rs:1796
11262023-12-06T17:42:40.550ZINFOSledAgent (ServiceManager): Setting up Nexus service file = sled-agent/src/services.rs:1840
11272023-12-06T17:42:40.550ZINFOSledAgent (ServiceManager): Ensuring address for OPTE port file = illumos-utils/src/running_zone.rs:652 zone = oxz_nexus_1dfcdbef-d660-4977-8363-0facffed5dd7
11282023-12-06T17:42:44.406ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::60fc:8ff:fe95:bcc8": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
11292023-12-06T17:42:45.339ZINFOSledAgent (ServiceManager): Writing ledger to /pool/int/a462a7f7-b628-40fe-80ff-4e4189e2d62b/config/.omicron-zones.json.tmp file = common/src/ledger.rs:198
11302023-12-06T17:42:45.340ZINFOSledAgent (ServiceManager): Writing ledger to /pool/int/b462a7f7-b628-40fe-80ff-4e4189e2d62b/config/.omicron-zones.json.tmp file = common/src/ledger.rs:198
11312023-12-06T17:42:45.341ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 51070918 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::1]:51442 req_id = 4184a24b-b3eb-495f-9ebb-a31dcae1abd7 response_code = 204 uri = /omicron-zones
11322023-12-06T17:42:45.344ZINFOSledAgent (RSS): Finished setting up services file = sled-agent/src/rack_setup/service.rs:1062
11332023-12-06T17:42:45.344ZINFOSledAgent (RSS): Writing ledger to /pool/int/a462a7f7-b628-40fe-80ff-4e4189e2d62b/config/.rss-plan-completed.marker.tmp file = common/src/ledger.rs:198
11342023-12-06T17:42:45.345ZINFOSledAgent (RSS): Writing ledger to /pool/int/b462a7f7-b628-40fe-80ff-4e4189e2d62b/config/.rss-plan-completed.marker.tmp file = common/src/ledger.rs:198
11352023-12-06T17:42:45.347ZINFOSledAgent (RSS): Handing off control to Nexus file = sled-agent/src/rack_setup/service.rs:549
11362023-12-06T17:42:45.347ZINFOSledAgent (RSS): Nexus address: [fd00:1122:3344:101::c]:12221 file = sled-agent/src/rack_setup/service.rs:551
11372023-12-06T17:42:45.379ZINFOSledAgent (RSS): rack_network_config: Some( RackNetworkConfigV1 { bgp: [], infra_ip_first: 10.151.2.174, infra_ip_last: 10.151.2.174, ports: [ PortConfigV1 { addresses: [ V4( Ipv4Network { addr: 10.151.2.174, prefix: 32, }, ), ], autoneg: false, bgp_peers: [], port: "qsfp0", routes: [ RouteConfig { destination: V4( Ipv4Network { addr: 0.0.0.0, prefix: 0, }, ), nexthop: 10.151.2.100, }, ], switch: Switch0, uplink_port_fec: None, uplink_port_speed: Speed40G, }, ], rack_subnet: Ipv6Network { addr: fd00:1122:3344:1::, prefix: 56, }, }, ) file = sled-agent/src/rack_setup/service.rs:658
11382023-12-06T17:42:45.379ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/racks/7487a3c5-0384-47e3-8028-3cee170ed4d7/initialization-complete): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/rack_setup/service.rs:679
11392023-12-06T17:42:45.510ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/racks/7487a3c5-0384-47e3-8028-3cee170ed4d7/initialization-complete): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/rack_setup/service.rs:679
11402023-12-06T17:42:45.656ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/racks/7487a3c5-0384-47e3-8028-3cee170ed4d7/initialization-complete): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/rack_setup/service.rs:679
11412023-12-06T17:42:45.853ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/racks/7487a3c5-0384-47e3-8028-3cee170ed4d7/initialization-complete): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/rack_setup/service.rs:679
11422023-12-06T17:42:47.203ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "929b6b33-b29f-4dae-b250-a5d14367587b", "content-length": "124", "date": "Wed, 06 Dec 2023 17:42:47 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "929b6b33-b29f-4dae-b250-a5d14367587b" } file = sled-agent/src/rack_setup/service.rs:679
11432023-12-06T17:42:47.580ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "c2f9b5be-1d17-42ff-86ea-d9a2a23a9bc7", "content-length": "124", "date": "Wed, 06 Dec 2023 17:42:47 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "c2f9b5be-1d17-42ff-86ea-d9a2a23a9bc7" } file = sled-agent/src/rack_setup/service.rs:679
11442023-12-06T17:42:47.956ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "e8d94037-e8e8-4473-bdff-f7aef705e665", "content-length": "124", "date": "Wed, 06 Dec 2023 17:42:47 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "e8d94037-e8e8-4473-bdff-f7aef705e665" } file = sled-agent/src/rack_setup/service.rs:679
11452023-12-06T17:42:48.211ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "4c55dd00-22ed-4545-9829-2dba3fad230e", "content-length": "124", "date": "Wed, 06 Dec 2023 17:42:47 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "4c55dd00-22ed-4545-9829-2dba3fad230e" } file = sled-agent/src/rack_setup/service.rs:679
11462023-12-06T17:42:48.546ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "091192ec-3884-482c-b4da-7e8365667c03", "content-length": "124", "date": "Wed, 06 Dec 2023 17:42:47 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "091192ec-3884-482c-b4da-7e8365667c03" } file = sled-agent/src/rack_setup/service.rs:679
11472023-12-06T17:42:48.919ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "4f7abdd6-6280-481e-9a61-30ddf556d703", "content-length": "124", "date": "Wed, 06 Dec 2023 17:42:48 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "4f7abdd6-6280-481e-9a61-30ddf556d703" } file = sled-agent/src/rack_setup/service.rs:679
11482023-12-06T17:42:49.408ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::60fc:8ff:fe95:bcc8": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
11492023-12-06T17:42:49.547ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "3848f0f4-4d21-427e-9bec-9357a35389cc", "content-length": "124", "date": "Wed, 06 Dec 2023 17:42:48 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "3848f0f4-4d21-427e-9bec-9357a35389cc" } file = sled-agent/src/rack_setup/service.rs:679
11502023-12-06T17:42:50.484ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "22cacb1b-dadb-4a04-b8c8-31394edad8b8", "content-length": "124", "date": "Wed, 06 Dec 2023 17:42:50 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "22cacb1b-dadb-4a04-b8c8-31394edad8b8" } file = sled-agent/src/rack_setup/service.rs:679
11512023-12-06T17:42:51.033ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "0899adeb-6950-4c71-9ecc-98a23aa48f03", "content-length": "124", "date": "Wed, 06 Dec 2023 17:42:50 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "0899adeb-6950-4c71-9ecc-98a23aa48f03" } file = sled-agent/src/rack_setup/service.rs:679
11522023-12-06T17:42:52.275ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "fe982f15-7aef-496c-beac-f272f7739459", "content-length": "124", "date": "Wed, 06 Dec 2023 17:42:52 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "fe982f15-7aef-496c-beac-f272f7739459" } file = sled-agent/src/rack_setup/service.rs:679
11532023-12-06T17:42:53.200ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "88945e56-74f6-4d53-8da5-1f5b011a87f8", "content-length": "124", "date": "Wed, 06 Dec 2023 17:42:53 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "88945e56-74f6-4d53-8da5-1f5b011a87f8" } file = sled-agent/src/rack_setup/service.rs:679
11542023-12-06T17:42:54.409ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::60fc:8ff:fe95:bcc8": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
11552023-12-06T17:42:54.578ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "49a52682-2f8d-4304-9106-6f9217107bf7", "content-length": "124", "date": "Wed, 06 Dec 2023 17:42:54 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "49a52682-2f8d-4304-9106-6f9217107bf7" } file = sled-agent/src/rack_setup/service.rs:679
11562023-12-06T17:42:55.528ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "067a7e5d-1f76-450a-825c-2d4c00718f01", "content-length": "124", "date": "Wed, 06 Dec 2023 17:42:55 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "067a7e5d-1f76-450a-825c-2d4c00718f01" } file = sled-agent/src/rack_setup/service.rs:679
11572023-12-06T17:42:58.198ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "901d9beb-eeb6-471b-b881-9a33213c50d4", "content-length": "124", "date": "Wed, 06 Dec 2023 17:42:58 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "901d9beb-eeb6-471b-b881-9a33213c50d4" } file = sled-agent/src/rack_setup/service.rs:679
11582023-12-06T17:42:59.410ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::60fc:8ff:fe95:bcc8": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
11592023-12-06T17:43:00.221ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "28038889-dd8e-4718-9811-6bbd04d145cd", "content-length": "124", "date": "Wed, 06 Dec 2023 17:43:00 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "28038889-dd8e-4718-9811-6bbd04d145cd" } file = sled-agent/src/rack_setup/service.rs:679
11602023-12-06T17:43:01.740ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "7d5ad7f0-a242-4488-9858-9c7ee71fb61d", "content-length": "124", "date": "Wed, 06 Dec 2023 17:43:01 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "7d5ad7f0-a242-4488-9858-9c7ee71fb61d" } file = sled-agent/src/rack_setup/service.rs:679
11612023-12-06T17:43:04.412ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::60fc:8ff:fe95:bcc8": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
11622023-12-06T17:43:05.415ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "ec6c1a01-f183-47b3-a115-1a984c03ee4e", "content-length": "124", "date": "Wed, 06 Dec 2023 17:43:05 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "ec6c1a01-f183-47b3-a115-1a984c03ee4e" } file = sled-agent/src/rack_setup/service.rs:679
11632023-12-06T17:43:06.319ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:769 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:33964
11642023-12-06T17:43:06.319ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 518 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:33964 req_id = 1adbc38f-88f4-49a0-ae15-6c874056d09f response_code = 200 uri = /baseboard
11652023-12-06T17:43:09.414ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::60fc:8ff:fe95:bcc8": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
11662023-12-06T17:43:09.889ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "d20dd3ff-74d1-4383-9390-96847b793485", "content-length": "124", "date": "Wed, 06 Dec 2023 17:43:09 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "d20dd3ff-74d1-4383-9390-96847b793485" } file = sled-agent/src/rack_setup/service.rs:679
11672023-12-06T17:43:13.209ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "00ba801c-b9bd-442a-8ed3-30f3a86fc027", "content-length": "124", "date": "Wed, 06 Dec 2023 17:43:13 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "00ba801c-b9bd-442a-8ed3-30f3a86fc027" } file = sled-agent/src/rack_setup/service.rs:679
11682023-12-06T17:43:14.415ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::60fc:8ff:fe95:bcc8": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
11692023-12-06T17:43:16.828ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "4a659ac9-342e-402a-92f4-38c45b4ec46e", "content-length": "124", "date": "Wed, 06 Dec 2023 17:43:16 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "4a659ac9-342e-402a-92f4-38c45b4ec46e" } file = sled-agent/src/rack_setup/service.rs:679
11702023-12-06T17:43:19.398ZWARNSledAgent (StorageMonitor): failed to notify nexus about Put(PhysicalDiskPutRequest { model: "synthetic-model", serial: "synthetic-serial-14b4dc87-ab46-49fb-a4b4-d361ae214c03", sled_id: 1e40a2ae-ffda-4a94-95f6-839f3349172b, variant: U2, vendor: "synthetic-vendor" }) err = "Error Response: status: 404 Not Found; headers: {\\"content-type\\": \\"application/json\\", \\"x-request-id\\": \\"72cec155-4675-44f7-825f-730fa83b7b97\\", \\"content-length\\": \\"173\\", \\"date\\": \\"Wed, 06 Dec 2023 17:43:19 GMT\\"}; value: Error { error_code: Some(\\"ObjectNotFound\\"), message: \\"not found: sled with id \\\\\\"1e40a2ae-ffda-4a94-95f6-839f3349172b\\\\\\"\\", request_id: \\"72cec155-4675-44f7-825f-730fa83b7b97\\" }" file = sled-agent/src/storage_monitor.rs:223 total duration = 157.565551897s
11712023-12-06T17:43:19.417ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::60fc:8ff:fe95:bcc8": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
11722023-12-06T17:43:19.600ZINFOSledAgent (HardwareMonitor): contacting server nexus, registering sled baseboard = Baseboard { part_number: "i86pc", revision: 0, serial_number: "c6eef3d4b5d9454d80e3983e3692cc58" } file = sled-agent/src/sled_agent.rs:629 id = 1e40a2ae-ffda-4a94-95f6-839f3349172b
11732023-12-06T17:43:19.645ZWARNSledAgent (StorageMonitor): failed to notify nexus about a new pool oxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03 on disk DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-f4b4dc87-ab46-49fb-a4b4-d361ae214c03", model: "synthetic-model" } err = "Error Response: status: 404 Not Found; headers: {\\"content-type\\": \\"application/json\\", \\"x-request-id\\": \\"598f6cfb-d01f-48ce-b4de-f8d5cbe4f56b\\", \\"content-length\\": \\"272\\", \\"date\\": \\"Wed, 06 Dec 2023 17:43:19 GMT\\"}; value: Error { error_code: Some(\\"ObjectNotFound\\"), message: \\"not found: physical-disk with id \\\\\\"vendor = \\\\\\"synthetic-vendor\\\\\\", serial = \\\\\\"synthetic-serial-f4b4dc87-ab46-49fb-a4b4-d361ae214c03\\\\\\", model = \\\\\\"synthetic-model\\\\\\"\\\\\\"\\", request_id: \\"598f6cfb-d01f-48ce-b4de-f8d5cbe4f56b\\" }" file = sled-agent/src/storage_monitor.rs:273 total duration = 157.808597242s
11742023-12-06T17:43:19.661ZWARNSledAgent (StorageMonitor): failed to notify nexus about a new pool oxp_24b4dc87-ab46-49fb-a4b4-d361ae214c03 on disk DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-24b4dc87-ab46-49fb-a4b4-d361ae214c03", model: "synthetic-model" } err = "Error Response: status: 404 Not Found; headers: {\\"content-type\\": \\"application/json\\", \\"x-request-id\\": \\"e8ef6d37-d3d1-4db1-835a-5f76478756f0\\", \\"content-length\\": \\"272\\", \\"date\\": \\"Wed, 06 Dec 2023 17:43:19 GMT\\"}; value: Error { error_code: Some(\\"ObjectNotFound\\"), message: \\"not found: physical-disk with id \\\\\\"vendor = \\\\\\"synthetic-vendor\\\\\\", serial = \\\\\\"synthetic-serial-24b4dc87-ab46-49fb-a4b4-d361ae214c03\\\\\\", model = \\\\\\"synthetic-model\\\\\\"\\\\\\"\\", request_id: \\"e8ef6d37-d3d1-4db1-835a-5f76478756f0\\" }" file = sled-agent/src/storage_monitor.rs:273 total duration = 157.82555106s
11752023-12-06T17:43:19.663ZWARNSledAgent (StorageMonitor): failed to notify nexus about a new pool oxp_ceb4461c-cf56-4719-ad3c-14430bfdfb60 on disk DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-ceb4461c-cf56-4719-ad3c-14430bfdfb60", model: "synthetic-model" } err = "Error Response: status: 404 Not Found; headers: {\\"content-type\\": \\"application/json\\", \\"x-request-id\\": \\"4fdccb75-cb50-4810-80c7-d09c0848fba0\\", \\"content-length\\": \\"272\\", \\"date\\": \\"Wed, 06 Dec 2023 17:43:19 GMT\\"}; value: Error { error_code: Some(\\"ObjectNotFound\\"), message: \\"not found: physical-disk with id \\\\\\"vendor = \\\\\\"synthetic-vendor\\\\\\", serial = \\\\\\"synthetic-serial-ceb4461c-cf56-4719-ad3c-14430bfdfb60\\\\\\", model = \\\\\\"synthetic-model\\\\\\"\\\\\\"\\", request_id: \\"4fdccb75-cb50-4810-80c7-d09c0848fba0\\" }" file = sled-agent/src/storage_monitor.rs:273 total duration = 157.827232304s
11762023-12-06T17:43:19.674ZWARNSledAgent (StorageMonitor): failed to notify nexus about a new pool oxi_a462a7f7-b628-40fe-80ff-4e4189e2d62b on disk DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-a462a7f7-b628-40fe-80ff-4e4189e2d62b", model: "synthetic-model" } err = "Error Response: status: 404 Not Found; headers: {\\"content-type\\": \\"application/json\\", \\"x-request-id\\": \\"c27fc767-f58c-435f-8518-364bb0018153\\", \\"content-length\\": \\"272\\", \\"date\\": \\"Wed, 06 Dec 2023 17:43:19 GMT\\"}; value: Error { error_code: Some(\\"ObjectNotFound\\"), message: \\"not found: physical-disk with id \\\\\\"vendor = \\\\\\"synthetic-vendor\\\\\\", serial = \\\\\\"synthetic-serial-a462a7f7-b628-40fe-80ff-4e4189e2d62b\\\\\\", model = \\\\\\"synthetic-model\\\\\\"\\\\\\"\\", request_id: \\"c27fc767-f58c-435f-8518-364bb0018153\\" }" file = sled-agent/src/storage_monitor.rs:273 total duration = 157.838644561s
11772023-12-06T17:43:19.676ZWARNSledAgent (StorageMonitor): failed to notify nexus about a new pool oxp_d462a7f7-b628-40fe-80ff-4e4189e2d62b on disk DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-d462a7f7-b628-40fe-80ff-4e4189e2d62b", model: "synthetic-model" } err = "Error Response: status: 404 Not Found; headers: {\\"content-type\\": \\"application/json\\", \\"x-request-id\\": \\"0854c34d-2185-4312-adde-65ef14533baf\\", \\"content-length\\": \\"272\\", \\"date\\": \\"Wed, 06 Dec 2023 17:43:19 GMT\\"}; value: Error { error_code: Some(\\"ObjectNotFound\\"), message: \\"not found: physical-disk with id \\\\\\"vendor = \\\\\\"synthetic-vendor\\\\\\", serial = \\\\\\"synthetic-serial-d462a7f7-b628-40fe-80ff-4e4189e2d62b\\\\\\", model = \\\\\\"synthetic-model\\\\\\"\\\\\\"\\", request_id: \\"0854c34d-2185-4312-adde-65ef14533baf\\" }" file = sled-agent/src/storage_monitor.rs:273 total duration = 157.839863376s
11782023-12-06T17:43:19.732ZWARNSledAgent (StorageMonitor): failed to notify nexus about a new pool oxi_b462a7f7-b628-40fe-80ff-4e4189e2d62b on disk DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-b462a7f7-b628-40fe-80ff-4e4189e2d62b", model: "synthetic-model" } err = "Error Response: status: 404 Not Found; headers: {\\"content-type\\": \\"application/json\\", \\"x-request-id\\": \\"8f79ee3c-1d0b-4c12-8d76-3aff8821b99d\\", \\"content-length\\": \\"272\\", \\"date\\": \\"Wed, 06 Dec 2023 17:43:19 GMT\\"}; value: Error { error_code: Some(\\"ObjectNotFound\\"), message: \\"not found: physical-disk with id \\\\\\"vendor = \\\\\\"synthetic-vendor\\\\\\", serial = \\\\\\"synthetic-serial-b462a7f7-b628-40fe-80ff-4e4189e2d62b\\\\\\", model = \\\\\\"synthetic-model\\\\\\"\\\\\\"\\", request_id: \\"8f79ee3c-1d0b-4c12-8d76-3aff8821b99d\\" }" file = sled-agent/src/storage_monitor.rs:273 total duration = 157.896051346s
11792023-12-06T17:43:19.737ZWARNSledAgent (StorageMonitor): failed to notify nexus about a new pool oxp_616b26df-e62a-4c68-b506-f4a923d8aaf7 on disk DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-616b26df-e62a-4c68-b506-f4a923d8aaf7", model: "synthetic-model" } err = "Error Response: status: 404 Not Found; headers: {\\"content-type\\": \\"application/json\\", \\"x-request-id\\": \\"6779ffca-00cd-4085-b76d-f70782f19008\\", \\"content-length\\": \\"272\\", \\"date\\": \\"Wed, 06 Dec 2023 17:43:19 GMT\\"}; value: Error { error_code: Some(\\"ObjectNotFound\\"), message: \\"not found: physical-disk with id \\\\\\"vendor = \\\\\\"synthetic-vendor\\\\\\", serial = \\\\\\"synthetic-serial-616b26df-e62a-4c68-b506-f4a923d8aaf7\\\\\\", model = \\\\\\"synthetic-model\\\\\\"\\\\\\"\\", request_id: \\"6779ffca-00cd-4085-b76d-f70782f19008\\" }" file = sled-agent/src/storage_monitor.rs:273 total duration = 157.901488554s
11802023-12-06T17:43:19.764ZWARNSledAgent (StorageMonitor): failed to notify nexus about a new pool oxp_14b4dc87-ab46-49fb-a4b4-d361ae214c03 on disk DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-14b4dc87-ab46-49fb-a4b4-d361ae214c03", model: "synthetic-model" } err = "Error Response: status: 404 Not Found; headers: {\\"content-type\\": \\"application/json\\", \\"x-request-id\\": \\"d434ce26-8e01-4be7-94cc-6bfafeb8a24a\\", \\"content-length\\": \\"272\\", \\"date\\": \\"Wed, 06 Dec 2023 17:43:19 GMT\\"}; value: Error { error_code: Some(\\"ObjectNotFound\\"), message: \\"not found: physical-disk with id \\\\\\"vendor = \\\\\\"synthetic-vendor\\\\\\", serial = \\\\\\"synthetic-serial-14b4dc87-ab46-49fb-a4b4-d361ae214c03\\\\\\", model = \\\\\\"synthetic-model\\\\\\"\\\\\\"\\", request_id: \\"d434ce26-8e01-4be7-94cc-6bfafeb8a24a\\" }" file = sled-agent/src/storage_monitor.rs:273 total duration = 157.928747867s
11812023-12-06T17:43:24.419ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::60fc:8ff:fe95:bcc8": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
11822023-12-06T17:43:25.775ZINFOSledAgent (StorageMonitor): Nexus notification complete: Put(PhysicalDiskPutRequest { model: "synthetic-model", serial: "synthetic-serial-14b4dc87-ab46-49fb-a4b4-d361ae214c03", sled_id: 1e40a2ae-ffda-4a94-95f6-839f3349172b, variant: U2, vendor: "synthetic-vendor" }) file = sled-agent/src/storage_monitor.rs:105
11832023-12-06T17:43:25.776ZINFOSledAgent (StorageMonitor): Nexus notification complete: Put(PhysicalDiskPutRequest { model: "synthetic-model", serial: "synthetic-serial-24b4dc87-ab46-49fb-a4b4-d361ae214c03", sled_id: 1e40a2ae-ffda-4a94-95f6-839f3349172b, variant: U2, vendor: "synthetic-vendor" }) file = sled-agent/src/storage_monitor.rs:105
11842023-12-06T17:43:25.776ZINFOSledAgent (StorageMonitor): Nexus notification complete: Put(PhysicalDiskPutRequest { model: "synthetic-model", serial: "synthetic-serial-31bd71cd-4736-4a12-a387-9b74b050396f", sled_id: 1e40a2ae-ffda-4a94-95f6-839f3349172b, variant: U2, vendor: "synthetic-vendor" }) file = sled-agent/src/storage_monitor.rs:105
11852023-12-06T17:43:25.776ZINFOSledAgent (StorageMonitor): Nexus notification complete: Put(PhysicalDiskPutRequest { model: "synthetic-model", serial: "synthetic-serial-616b26df-e62a-4c68-b506-f4a923d8aaf7", sled_id: 1e40a2ae-ffda-4a94-95f6-839f3349172b, variant: U2, vendor: "synthetic-vendor" }) file = sled-agent/src/storage_monitor.rs:105
11862023-12-06T17:43:25.776ZINFOSledAgent (StorageMonitor): Nexus notification complete: Put(PhysicalDiskPutRequest { model: "synthetic-model", serial: "synthetic-serial-a462a7f7-b628-40fe-80ff-4e4189e2d62b", sled_id: 1e40a2ae-ffda-4a94-95f6-839f3349172b, variant: M2, vendor: "synthetic-vendor" }) file = sled-agent/src/storage_monitor.rs:105
11872023-12-06T17:43:25.776ZINFOSledAgent (StorageMonitor): Nexus notification complete: Put(PhysicalDiskPutRequest { model: "synthetic-model", serial: "synthetic-serial-b462a7f7-b628-40fe-80ff-4e4189e2d62b", sled_id: 1e40a2ae-ffda-4a94-95f6-839f3349172b, variant: M2, vendor: "synthetic-vendor" }) file = sled-agent/src/storage_monitor.rs:105
11882023-12-06T17:43:25.776ZINFOSledAgent (StorageMonitor): Nexus notification complete: Put(PhysicalDiskPutRequest { model: "synthetic-model", serial: "synthetic-serial-cd70d7f6-2354-4bf2-8012-55bf9eaf7930", sled_id: 1e40a2ae-ffda-4a94-95f6-839f3349172b, variant: U2, vendor: "synthetic-vendor" }) file = sled-agent/src/storage_monitor.rs:105
11892023-12-06T17:43:25.776ZINFOSledAgent (StorageMonitor): Nexus notification complete: Put(PhysicalDiskPutRequest { model: "synthetic-model", serial: "synthetic-serial-ceb4461c-cf56-4719-ad3c-14430bfdfb60", sled_id: 1e40a2ae-ffda-4a94-95f6-839f3349172b, variant: U2, vendor: "synthetic-vendor" }) file = sled-agent/src/storage_monitor.rs:105
11902023-12-06T17:43:25.776ZINFOSledAgent (StorageMonitor): Nexus notification complete: Put(PhysicalDiskPutRequest { model: "synthetic-model", serial: "synthetic-serial-d462a7f7-b628-40fe-80ff-4e4189e2d62b", sled_id: 1e40a2ae-ffda-4a94-95f6-839f3349172b, variant: U2, vendor: "synthetic-vendor" }) file = sled-agent/src/storage_monitor.rs:105
11912023-12-06T17:43:25.776ZINFOSledAgent (StorageMonitor): Nexus notification complete: Put(PhysicalDiskPutRequest { model: "synthetic-model", serial: "synthetic-serial-e4b4dc87-ab46-49fb-a4b4-d361ae214c03", sled_id: 1e40a2ae-ffda-4a94-95f6-839f3349172b, variant: U2, vendor: "synthetic-vendor" }) file = sled-agent/src/storage_monitor.rs:105
11922023-12-06T17:43:25.776ZINFOSledAgent (StorageMonitor): Nexus notification complete: Put(PhysicalDiskPutRequest { model: "synthetic-model", serial: "synthetic-serial-f4b4dc87-ab46-49fb-a4b4-d361ae214c03", sled_id: 1e40a2ae-ffda-4a94-95f6-839f3349172b, variant: U2, vendor: "synthetic-vendor" }) file = sled-agent/src/storage_monitor.rs:105
11932023-12-06T17:43:26.408ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 404 Not Found; headers: {"content-type": "application/json", "x-request-id": "60004b14-8fe1-42bb-86b7-737614015a8b", "content-length": "174", "date": "Wed, 06 Dec 2023 17:43:25 GMT"}; value: Error { error_code: Some("ObjectNotFound"), message: "not found: zpool with id \"14b4dc87-ab46-49fb-a4b4-d361ae214c03\"", request_id: "60004b14-8fe1-42bb-86b7-737614015a8b" } file = sled-agent/src/rack_setup/service.rs:679
11942023-12-06T17:43:27.698ZINFOSledAgent (StorageMonitor): Nexus notification complete: ZpoolPutRequest { disk_model: "synthetic-model", disk_serial: "synthetic-serial-14b4dc87-ab46-49fb-a4b4-d361ae214c03", disk_vendor: "synthetic-vendor", size: ByteCount(20937965568) } file = sled-agent/src/storage_monitor.rs:105
11952023-12-06T17:43:27.698ZINFOSledAgent (StorageMonitor): Nexus notification complete: ZpoolPutRequest { disk_model: "synthetic-model", disk_serial: "synthetic-serial-24b4dc87-ab46-49fb-a4b4-d361ae214c03", disk_vendor: "synthetic-vendor", size: ByteCount(20937965568) } file = sled-agent/src/storage_monitor.rs:105
11962023-12-06T17:43:27.698ZINFOSledAgent (StorageMonitor): Nexus notification complete: ZpoolPutRequest { disk_model: "synthetic-model", disk_serial: "synthetic-serial-31bd71cd-4736-4a12-a387-9b74b050396f", disk_vendor: "synthetic-vendor", size: ByteCount(20937965568) } file = sled-agent/src/storage_monitor.rs:105
11972023-12-06T17:43:27.699ZINFOSledAgent (StorageMonitor): Nexus notification complete: ZpoolPutRequest { disk_model: "synthetic-model", disk_serial: "synthetic-serial-616b26df-e62a-4c68-b506-f4a923d8aaf7", disk_vendor: "synthetic-vendor", size: ByteCount(20937965568) } file = sled-agent/src/storage_monitor.rs:105
11982023-12-06T17:43:27.699ZINFOSledAgent (StorageMonitor): Nexus notification complete: ZpoolPutRequest { disk_model: "synthetic-model", disk_serial: "synthetic-serial-a462a7f7-b628-40fe-80ff-4e4189e2d62b", disk_vendor: "synthetic-vendor", size: ByteCount(20937965568) } file = sled-agent/src/storage_monitor.rs:105
11992023-12-06T17:43:27.699ZINFOSledAgent (StorageMonitor): Nexus notification complete: ZpoolPutRequest { disk_model: "synthetic-model", disk_serial: "synthetic-serial-b462a7f7-b628-40fe-80ff-4e4189e2d62b", disk_vendor: "synthetic-vendor", size: ByteCount(20937965568) } file = sled-agent/src/storage_monitor.rs:105
12002023-12-06T17:43:27.699ZINFOSledAgent (StorageMonitor): Nexus notification complete: ZpoolPutRequest { disk_model: "synthetic-model", disk_serial: "synthetic-serial-cd70d7f6-2354-4bf2-8012-55bf9eaf7930", disk_vendor: "synthetic-vendor", size: ByteCount(20937965568) } file = sled-agent/src/storage_monitor.rs:105
12012023-12-06T17:43:27.699ZINFOSledAgent (StorageMonitor): Nexus notification complete: ZpoolPutRequest { disk_model: "synthetic-model", disk_serial: "synthetic-serial-ceb4461c-cf56-4719-ad3c-14430bfdfb60", disk_vendor: "synthetic-vendor", size: ByteCount(20937965568) } file = sled-agent/src/storage_monitor.rs:105
12022023-12-06T17:43:27.699ZINFOSledAgent (StorageMonitor): Nexus notification complete: ZpoolPutRequest { disk_model: "synthetic-model", disk_serial: "synthetic-serial-d462a7f7-b628-40fe-80ff-4e4189e2d62b", disk_vendor: "synthetic-vendor", size: ByteCount(20937965568) } file = sled-agent/src/storage_monitor.rs:105
12032023-12-06T17:43:27.699ZINFOSledAgent (StorageMonitor): Nexus notification complete: ZpoolPutRequest { disk_model: "synthetic-model", disk_serial: "synthetic-serial-e4b4dc87-ab46-49fb-a4b4-d361ae214c03", disk_vendor: "synthetic-vendor", size: ByteCount(20937965568) } file = sled-agent/src/storage_monitor.rs:105
12042023-12-06T17:43:27.699ZINFOSledAgent (StorageMonitor): Nexus notification complete: ZpoolPutRequest { disk_model: "synthetic-model", disk_serial: "synthetic-serial-f4b4dc87-ab46-49fb-a4b4-d361ae214c03", disk_vendor: "synthetic-vendor", size: ByteCount(20937965568) } file = sled-agent/src/storage_monitor.rs:105
12052023-12-06T17:43:29.420ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::60fc:8ff:fe95:bcc8": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
12062023-12-06T17:43:34.422ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::60fc:8ff:fe95:bcc8": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
12072023-12-06T17:43:36.353ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:769 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:36874
12082023-12-06T17:43:36.354ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 743 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:36874 req_id = e1956ee6-f348-464e-aa2f-c4ec3dd02eb0 response_code = 200 uri = /baseboard
12092023-12-06T17:43:37.583ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:769 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:36397
12102023-12-06T17:43:37.584ZINFOSledAgent (PortManager): Ensuring VPC firewall rules file = illumos-utils/src/opte/port_manager.rs:399 rules = [VpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 3a6a2032-caca-49e5-b81c-96d8b3f5cbfe, kind: Service { id: f4bd4239-a995-46ad-bd81-8c0c7f2f9049 }, name: Name("external-dns-f4bd4239-a995-46ad-bd81-8c0c7f2f9049"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 210, 64])), subnet: V4(Ipv4Net(Ipv4Network { addr: 172.30.1.0, prefix: 24 })), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: 58e4c4d4-d4dc-499f-91ef-477be3fc8fbf, kind: Service { id: 2cfabf1e-5b74-40a8-8d00-5f984444bf51 }, name: Name("external-dns-2cfabf1e-5b74-40a8-8d00-5f984444bf51"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 196, 173])), subnet: V4(Ipv4Net(Ipv4Network { addr: 172.30.1.0, prefix: 24 })), vni: Vni(100), primary: true, slot: 1 }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, VpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 43f4b26a-16ba-4087-909c-b7a0873aa264, kind: Service { id: d2016ddd-0efc-4439-89aa-2f166d289e6f }, name: Name("nexus-d2016ddd-0efc-4439-89aa-2f166d289e6f"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 141, 175])), subnet: V4(Ipv4Net(Ipv4Network { addr: 172.30.2.0, prefix: 24 })), vni: Vni(100), primary: true, slot: 1 }, NetworkInterface { id: 5465fc8d-98dc-4a8e-8ee0-96d7fdc4ec73, kind: Service { id: 9a25727f-e844-4dd1-b1b4-978eca7f5c90 }, name: Name("nexus-9a25727f-e844-4dd1-b1b4-978eca7f5c90"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 156, 174])), subnet: V4(Ipv4Net(Ipv4Network { addr: 172.30.2.0, prefix: 24 })), vni: Vni(100), primary: true, slot: 1 }, NetworkInterface { id: 757a1ebf-4b4a-4ea0-a6c4-be5a8336ba90, kind: Service { id: 1dfcdbef-d660-4977-8363-0facffed5dd7 }, name: Name("nexus-1dfcdbef-d660-4977-8363-0facffed5dd7"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 248, 89])), subnet: V4(Ipv4Net(Ipv4Network { addr: 172.30.2.0, prefix: 24 })), vni: Vni(100), primary: true, slot: 1 }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }] vni = Vni(100)
12112023-12-06T17:43:37.584ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:418 port = "opte0" rules = []
12122023-12-06T17:43:37.584ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:418 port = "opte4" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
12132023-12-06T17:43:37.584ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:418 port = "opte2" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
12142023-12-06T17:43:37.584ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:418 port = "opte1" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
12152023-12-06T17:43:37.584ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:418 port = "opte3" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
12162023-12-06T17:43:37.584ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:418 port = "opte5" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
12172023-12-06T17:43:37.584ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 342 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:36397 req_id = 4dc414e7-e27a-4e14-9cd0-724851aa9e90 response_code = 204 uri = /vpc/001de000-074c-4000-8000-000000000000/firewall/rules
12182023-12-06T17:43:37.997ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:769 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:55558
12192023-12-06T17:43:37.998ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 85 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:55558 req_id = 1a635a3e-8f83-4a2a-90f0-7f6cfd465c30 response_code = 200 uri = /network-bootstore-config
12202023-12-06T17:43:38.028ZINFOSledAgent (RSS): Handoff to Nexus is complete file = sled-agent/src/rack_setup/service.rs:689
12212023-12-06T17:43:39.422ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::60fc:8ff:fe95:bcc8": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
12222023-12-06T17:43:44.424ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::60fc:8ff:fe95:bcc8": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
12232023-12-06T17:43:49.425ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::60fc:8ff:fe95:bcc8": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
12242023-12-06T17:43:49.517ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:769 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:52875
12252023-12-06T17:43:49.518ZINFOSledAgent (InstanceManager): ensuring instance is registered file = sled-agent/src/instance_manager.rs:227 hardware = InstanceHardware { properties: InstanceProperties { ncpus: InstanceCpuCount(2), memory: ByteCount(1073741824), hostname: "localshark" }, nics: [NetworkInterface { id: ab76162d-02c5-4805-8660-6d3f946e2148, kind: Instance { id: 798fd184-57e7-4c63-a223-b0a810e00937 }, name: Name("net0"), ip: 172.30.0.5, mac: MacAddr(MacAddr6([168, 64, 37, 244, 210, 95])), subnet: V4(Ipv4Net(Ipv4Network { addr: 172.30.0.0, prefix: 22 })), vni: Vni(7967576), primary: true, slot: 0 }], source_nat: SourceNatConfig { ip: 10.151.2.175, first_port: 0, last_port: 16383 }, external_ips: [10.151.2.176], firewall_rules: [VpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: ab76162d-02c5-4805-8660-6d3f946e2148, kind: Instance { id: 798fd184-57e7-4c63-a223-b0a810e00937 }, name: Name("net0"), ip: 172.30.0.5, mac: MacAddr(MacAddr6([168, 64, 37, 244, 210, 95])), subnet: V4(Ipv4Net(Ipv4Network { addr: 172.30.0.0, prefix: 22 })), vni: Vni(7967576), primary: true, slot: 0 }], filter_hosts: None, filter_ports: None, filter_protocols: Some([Icmp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, VpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: ab76162d-02c5-4805-8660-6d3f946e2148, kind: Instance { id: 798fd184-57e7-4c63-a223-b0a810e00937 }, name: Name("net0"), ip: 172.30.0.5, mac: MacAddr(MacAddr6([168, 64, 37, 244, 210, 95])), subnet: V4(Ipv4Net(Ipv4Network { addr: 172.30.0.0, prefix: 22 })), vni: Vni(7967576), primary: true, slot: 0 }], filter_hosts: Some([Vpc(Vni(7967576))]), filter_ports: None, filter_protocols: None, action: Allow, priority: VpcFirewallRulePriority(65534) }, VpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: ab76162d-02c5-4805-8660-6d3f946e2148, kind: Instance { id: 798fd184-57e7-4c63-a223-b0a810e00937 }, name: Name("net0"), ip: 172.30.0.5, mac: MacAddr(MacAddr6([168, 64, 37, 244, 210, 95])), subnet: V4(Ipv4Net(Ipv4Network { addr: 172.30.0.0, prefix: 22 })), vni: Vni(7967576), primary: true, slot: 0 }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(22), last: L4Port(22) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }], dhcp_config: DhcpConfig { dns_servers: [1.1.1.1, 9.9.9.9], host_domain: None, search_domains: [] }, disks: [DiskRequest { device: "nvme", name: "disk-10889371ea3", read_only: false, slot: Slot(1), volume_construction_request: Volume { block_size: 512, id: ffb3dff6-c782-4ce7-800e-13770926bc89, read_only_parent: Some(Url { block_size: 512, id: 4bc1bf58-40f0-46cc-a5ac-f90b12ff5116, url: "http://[fd00:1122:3344:101::1]:54321/debian-11-genericcloud-amd64.raw" }), sub_volumes: [Region { block_size: 512, blocks_per_extent: 131072, extent_count: 32, gen: 1, opts: CrucibleOpts { cert_pem: None, control: None, flush_timeout: None, id: ffb3dff6-c782-4ce7-800e-13770926bc89, key: Some("cKWBBvAjYkTD3jSFxo6Arz79k+dVn6SpvhQyFV1RXnc="), key_pem: None, lossy: false, read_only: false, root_cert_pem: None, target: ["[fd00:1122:3344:101::12]:19000", "[fd00:1122:3344:101::17]:19000", "[fd00:1122:3344:101::19]:19000"] } }] } }], cloud_init_bytes: Some("6zyQTVNXSU40LjEAAgEBAAIAAioA+AEAKgBAAAAAAAAAAAAAAAApeFY0EmNpZGF0YSAgICAgRkFUMTIgICAOH75bfKwiwHQLVrQOuwcAzRBe6/Ay5M0WzRnr/lRoaXMgaXMgbm90IGEgYm9vdGFibGUgZGlzay4gIFBsZWFzZSBpbnNlcnQgYSBib290YWJsZSBmbG9wcHkgYW5kDQpwcmVzcyBhbnkga2V5IHRvIHRyeSBhZ2FpbiAuLi4gDQoAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAVar4////DwAAAAAAAAAA8P//////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////D/j///8PAAAAAAAAAADw//////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////8PY2lkYXRhICAgICAIAAAAAAAAAAAAAAAAAAAAAAAAAABBbQBlAHQAYQAtAA8A1GQAYQB0AGEAAAD//wAA/////01FVEEtRH4xICAgAACUeI2GV4ZXAAB4jYZXAgC3AAAAQXUAcwBlAHIALQAPALBkAGEAdABhAAAA//8AAP////9VU0VSLUR+MSAgIAAAlHiNhleGVwAAeI2GVwAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAHsiaW5zdGFuY2UtaWQiOiI3OThmZDE4NC01N2U3LTRjNjMtYTIyMy1iMGE4MTBlMDA5MzciLCJsb2NhbC1ob3N0bmFtZSI6ImxvY2Fsc2hhcmsiLCJwdWJsaWMta2V5cyI6WyJzc2gtZWQyNTUxOSBBQUFBQzNOemFDMWxaREkxTlRFNUFBQUFJRmtLdE1mZi9KNEh1N1pORmRMdHMwQmFoaThlRmJ6UExJL283TnN0VDV6dyJdfQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA") } instance_id = 798fd184-57e7-4c63-a223-b0a810e00937 instance_runtime = InstanceRuntimeState { propolis_id: Some(ef02d4e7-7031-4005-b456-539acbeb1df5), dst_propolis_id: None, migration_id: None, gen: Generation(3), time_updated: 2023-12-06T17:43:49.039589950Z } propolis_addr = [fd00:1122:3344:101::22]:12400 propolis_id = ef02d4e7-7031-4005-b456-539acbeb1df5 vmm_runtime = VmmRuntimeState { state: Starting, gen: Generation(1), time_updated: 2023-12-06T17:43:48.980976Z }
12262023-12-06T17:43:49.518ZINFOSledAgent (InstanceManager): registering new instance file = sled-agent/src/instance_manager.rs:261 instance_id = 798fd184-57e7-4c63-a223-b0a810e00937
12272023-12-06T17:43:49.518ZINFOSledAgent (InstanceManager): initializing new Instance file = sled-agent/src/instance.rs:599 instance_id = 798fd184-57e7-4c63-a223-b0a810e00937 propolis_id = ef02d4e7-7031-4005-b456-539acbeb1df5 state = InstanceInitialState { hardware: InstanceHardware { properties: InstanceProperties { ncpus: InstanceCpuCount(2), memory: ByteCount(1073741824), hostname: "localshark" }, nics: [NetworkInterface { id: ab76162d-02c5-4805-8660-6d3f946e2148, kind: Instance { id: 798fd184-57e7-4c63-a223-b0a810e00937 }, name: Name("net0"), ip: 172.30.0.5, mac: MacAddr(MacAddr6([168, 64, 37, 244, 210, 95])), subnet: V4(Ipv4Net(Ipv4Network { addr: 172.30.0.0, prefix: 22 })), vni: Vni(7967576), primary: true, slot: 0 }], source_nat: SourceNatConfig { ip: 10.151.2.175, first_port: 0, last_port: 16383 }, external_ips: [10.151.2.176], firewall_rules: [VpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: ab76162d-02c5-4805-8660-6d3f946e2148, kind: Instance { id: 798fd184-57e7-4c63-a223-b0a810e00937 }, name: Name("net0"), ip: 172.30.0.5, mac: MacAddr(MacAddr6([168, 64, 37, 244, 210, 95])), subnet: V4(Ipv4Net(Ipv4Network { addr: 172.30.0.0, prefix: 22 })), vni: Vni(7967576), primary: true, slot: 0 }], filter_hosts: None, filter_ports: None, filter_protocols: Some([Icmp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, VpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: ab76162d-02c5-4805-8660-6d3f946e2148, kind: Instance { id: 798fd184-57e7-4c63-a223-b0a810e00937 }, name: Name("net0"), ip: 172.30.0.5, mac: MacAddr(MacAddr6([168, 64, 37, 244, 210, 95])), subnet: V4(Ipv4Net(Ipv4Network { addr: 172.30.0.0, prefix: 22 })), vni: Vni(7967576), primary: true, slot: 0 }], filter_hosts: Some([Vpc(Vni(7967576))]), filter_ports: None, filter_protocols: None, action: Allow, priority: VpcFirewallRulePriority(65534) }, VpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: ab76162d-02c5-4805-8660-6d3f946e2148, kind: Instance { id: 798fd184-57e7-4c63-a223-b0a810e00937 }, name: Name("net0"), ip: 172.30.0.5, mac: MacAddr(MacAddr6([168, 64, 37, 244, 210, 95])), subnet: V4(Ipv4Net(Ipv4Network { addr: 172.30.0.0, prefix: 22 })), vni: Vni(7967576), primary: true, slot: 0 }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(22), last: L4Port(22) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }], dhcp_config: DhcpConfig { dns_servers: [1.1.1.1, 9.9.9.9], host_domain: None, search_domains: [] }, disks: [DiskRequest { device: "nvme", name: "disk-10889371ea3", read_only: false, slot: Slot(1), volume_construction_request: Volume { block_size: 512, id: ffb3dff6-c782-4ce7-800e-13770926bc89, read_only_parent: Some(Url { block_size: 512, id: 4bc1bf58-40f0-46cc-a5ac-f90b12ff5116, url: "http://[fd00:1122:3344:101::1]:54321/debian-11-genericcloud-amd64.raw" }), sub_volumes: [Region { block_size: 512, blocks_per_extent: 131072, extent_count: 32, gen: 1, opts: CrucibleOpts { cert_pem: None, control: None, flush_timeout: None, id: ffb3dff6-c782-4ce7-800e-13770926bc89, key: Some("cKWBBvAjYkTD3jSFxo6Arz79k+dVn6SpvhQyFV1RXnc="), key_pem: None, lossy: false, read_only: false, root_cert_pem: None, target: ["[fd00:1122:3344:101::12]:19000", "[fd00:1122:3344:101::17]:19000", "[fd00:1122:3344:101::19]:19000"] } }] } }], cloud_init_bytes: Some("6zyQTVNXSU40LjEAAgEBAAIAAioA+AEAKgBAAAAAAAAAAAAAAAApeFY0EmNpZGF0YSAgICAgRkFUMTIgICAOH75bfKwiwHQLVrQOuwcAzRBe6/Ay5M0WzRnr/lRoaXMgaXMgbm90IGEgYm9vdGFibGUgZGlzay4gIFBsZWFzZSBpbnNlcnQgYSBib290YWJsZSBmbG9wcHkgYW5kDQpwcmVzcyBhbnkga2V5IHRvIHRyeSBhZ2FpbiAuLi4gDQoAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAVar4////DwAAAAAAAAAA8P//////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////D/j///8PAAAAAAAAAADw//////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////8PY2lkYXRhICAgICAIAAAAAAAAAAAAAAAAAAAAAAAAAABBbQBlAHQAYQAtAA8A1GQAYQB0AGEAAAD//wAA/////01FVEEtRH4xICAgAACUeI2GV4ZXAAB4jYZXAgC3AAAAQXUAcwBlAHIALQAPALBkAGEAdABhAAAA//8AAP////9VU0VSLUR+MSAgIAAAlHiNhleGVwAAeI2GVwAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAHsiaW5zdGFuY2UtaWQiOiI3OThmZDE4NC01N2U3LTRjNjMtYTIyMy1iMGE4MTBlMDA5MzciLCJsb2NhbC1ob3N0bmFtZSI6ImxvY2Fsc2hhcmsiLCJwdWJsaWMta2V5cyI6WyJzc2gtZWQyNTUxOSBBQUFBQzNOemFDMWxaREkxTlRFNUFBQUFJRmtLdE1mZi9KNEh1N1pORmRMdHMwQmFoaThlRmJ6UExJL283TnN0VDV6dyJdfQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA") }, instance_runtime: InstanceRuntimeState { propolis_id: Some(ef02d4e7-7031-4005-b456-539acbeb1df5), dst_propolis_id: None, migration_id: None, gen: Generation(3), time_updated: 2023-12-06T17:43:49.039589950Z }, vmm_runtime: VmmRuntimeState { state: Starting, gen: Generation(1), time_updated: 2023-12-06T17:43:48.980976Z }, propolis_addr: [fd00:1122:3344:101::22]:12400 }
12282023-12-06T17:43:49.518ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 1204 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:52875 req_id = 7191ac06-676f-4bf3-a464-00a508ec72a1 response_code = 200 uri = /instances/798fd184-57e7-4c63-a223-b0a810e00937
12292023-12-06T17:43:49.629ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:769 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:47331
12302023-12-06T17:43:49.651ZINFOSledAgent (PortManager): Created OPTE port file = illumos-utils/src/opte/port_manager.rs:383 port = Port { inner: PortInner { name: "opte6", _ip: 172.30.0.5, mac: MacAddr6([168, 64, 37, 244, 210, 95]), slot: 0, vni: Vni { inner: 7967576 }, gateway: Gateway { mac: MacAddr6([168, 64, 37, 255, 119, 119]), ip: 172.30.0.1 }, vnic: "vopte6" } }
12312023-12-06T17:43:49.651ZINFOSledAgent (StorageManager): Received GetLatestResources(Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) }) file = sled-storage/src/manager.rs:346
12322023-12-06T17:43:49.686ZINFOSledAgent (InstanceManager): Configuring new Omicron zone: oxz_propolis-server_ef02d4e7-7031-4005-b456-539acbeb1df5 file = illumos-utils/src/zone.rs:314 instance_id = 798fd184-57e7-4c63-a223-b0a810e00937
12332023-12-06T17:43:49.708ZINFOSledAgent (InstanceManager): Installing Omicron zone: oxz_propolis-server_ef02d4e7-7031-4005-b456-539acbeb1df5 file = illumos-utils/src/zone.rs:353 instance_id = 798fd184-57e7-4c63-a223-b0a810e00937
12342023-12-06T17:43:49.907ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:769 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::d]:52761
12352023-12-06T17:43:49.908ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 1920 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::d]:52761 req_id = 936787c0-9200-4899-9a53-9caf0b78203a response_code = 200 uri = /metrics/collect/1e40a2ae-ffda-4a94-95f6-839f3349172b
12362023-12-06T17:43:51.208ZINFOSledAgent (InstanceManager): Profile for oxz_propolis-server_ef02d4e7-7031-4005-b456-539acbeb1df5: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="system/illumos/propolis-server"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value="oxControlInstance0"/> <propval type="astring" name="gateway" value="fd00:1122:3344:101::1"/> <propval type="astring" name="listen_addr" value="fd00:1122:3344:101::22"/> <propval type="astring" name="listen_port" value="12400"/> <propval type="astring" name="metric_addr" value="[fd00:1122:3344:101::a]:12221"/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34 instance_id = 798fd184-57e7-4c63-a223-b0a810e00937
12372023-12-06T17:43:51.209ZINFOSledAgent (InstanceManager): Zone booting file = illumos-utils/src/running_zone.rs:512 instance_id = 798fd184-57e7-4c63-a223-b0a810e00937 zone = oxz_propolis-server_ef02d4e7-7031-4005-b456-539acbeb1df5
12382023-12-06T17:43:51.386ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_ef02d4e7-7031-4005-b456-539acbeb1df5") failed: Property not found. retry in 48.214624ms file = illumos-utils/src/svc.rs:36 instance_id = 798fd184-57e7-4c63-a223-b0a810e00937 zone = oxz_propolis-server_ef02d4e7-7031-4005-b456-539acbeb1df5
12392023-12-06T17:43:51.443ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_ef02d4e7-7031-4005-b456-539acbeb1df5") failed: Property not found. retry in 104.88077ms file = illumos-utils/src/svc.rs:36 instance_id = 798fd184-57e7-4c63-a223-b0a810e00937 zone = oxz_propolis-server_ef02d4e7-7031-4005-b456-539acbeb1df5
12402023-12-06T17:43:51.611ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_ef02d4e7-7031-4005-b456-539acbeb1df5") failed: Property not found. retry in 210.001066ms file = illumos-utils/src/svc.rs:36 instance_id = 798fd184-57e7-4c63-a223-b0a810e00937 zone = oxz_propolis-server_ef02d4e7-7031-4005-b456-539acbeb1df5
12412023-12-06T17:43:51.831ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_ef02d4e7-7031-4005-b456-539acbeb1df5") failed: Property not found. retry in 425.432251ms file = illumos-utils/src/svc.rs:36 instance_id = 798fd184-57e7-4c63-a223-b0a810e00937 zone = oxz_propolis-server_ef02d4e7-7031-4005-b456-539acbeb1df5
12422023-12-06T17:43:52.267ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_ef02d4e7-7031-4005-b456-539acbeb1df5") failed: Property not found. retry in 401.898577ms file = illumos-utils/src/svc.rs:36 instance_id = 798fd184-57e7-4c63-a223-b0a810e00937 zone = oxz_propolis-server_ef02d4e7-7031-4005-b456-539acbeb1df5
12432023-12-06T17:43:52.680ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_ef02d4e7-7031-4005-b456-539acbeb1df5") failed: Property not found. retry in 952.301512ms file = illumos-utils/src/svc.rs:36 instance_id = 798fd184-57e7-4c63-a223-b0a810e00937 zone = oxz_propolis-server_ef02d4e7-7031-4005-b456-539acbeb1df5
12442023-12-06T17:43:53.643ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_ef02d4e7-7031-4005-b456-539acbeb1df5") failed: Property not found. retry in 598.027541ms file = illumos-utils/src/svc.rs:36 instance_id = 798fd184-57e7-4c63-a223-b0a810e00937 zone = oxz_propolis-server_ef02d4e7-7031-4005-b456-539acbeb1df5
12452023-12-06T17:43:54.251ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_ef02d4e7-7031-4005-b456-539acbeb1df5") failed: Property not found. retry in 1.112123092s file = illumos-utils/src/svc.rs:36 instance_id = 798fd184-57e7-4c63-a223-b0a810e00937 zone = oxz_propolis-server_ef02d4e7-7031-4005-b456-539acbeb1df5
12462023-12-06T17:43:54.426ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::60fc:8ff:fe95:bcc8": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
12472023-12-06T17:43:55.373ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_ef02d4e7-7031-4005-b456-539acbeb1df5") failed: Property not found. retry in 1.416561237s file = illumos-utils/src/svc.rs:36 instance_id = 798fd184-57e7-4c63-a223-b0a810e00937 zone = oxz_propolis-server_ef02d4e7-7031-4005-b456-539acbeb1df5
12482023-12-06T17:43:56.821ZINFOSledAgent (InstanceManager): Started propolis in zone: oxz_propolis-server_ef02d4e7-7031-4005-b456-539acbeb1df5 file = sled-agent/src/instance.rs:986 instance_id = 798fd184-57e7-4c63-a223-b0a810e00937
12492023-12-06T17:43:56.830ZINFOSledAgent (InstanceManager): Propolis SMF service is online file = sled-agent/src/instance.rs:995 instance_id = 798fd184-57e7-4c63-a223-b0a810e00937
12502023-12-06T17:43:56.864ZINFOSledAgent (InstanceManager): Propolis HTTP server online file = sled-agent/src/instance.rs:1009 instance_id = 798fd184-57e7-4c63-a223-b0a810e00937
12512023-12-06T17:43:56.865ZINFOSledAgent (InstanceManager): Sending ensure request to propolis: InstanceEnsureRequest { cloud_init_bytes: Some("6zyQTVNXSU40LjEAAgEBAAIAAioA+AEAKgBAAAAAAAAAAAAAAAApeFY0EmNpZGF0YSAgICAgRkFUMTIgICAOH75bfKwiwHQLVrQOuwcAzRBe6/Ay5M0WzRnr/lRoaXMgaXMgbm90IGEgYm9vdGFibGUgZGlzay4gIFBsZWFzZSBpbnNlcnQgYSBib290YWJsZSBmbG9wcHkgYW5kDQpwcmVzcyBhbnkga2V5IHRvIHRyeSBhZ2FpbiAuLi4gDQoAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAVar4////DwAAAAAAAAAA8P//////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////D/j///8PAAAAAAAAAADw//////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////8PY2lkYXRhICAgICAIAAAAAAAAAAAAAAAAAAAAAAAAAABBbQBlAHQAYQAtAA8A1GQAYQB0AGEAAAD//wAA/////01FVEEtRH4xICAgAACUeI2GV4ZXAAB4jYZXAgC3AAAAQXUAcwBlAHIALQAPALBkAGEAdABhAAAA//8AAP////9VU0VSLUR+MSAgIAAAlHiNhleGVwAAeI2GVwAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAHsiaW5zdGFuY2UtaWQiOiI3OThmZDE4NC01N2U3LTRjNjMtYTIyMy1iMGE4MTBlMDA5MzciLCJsb2NhbC1ob3N0bmFtZSI6ImxvY2Fsc2hhcmsiLCJwdWJsaWMta2V5cyI6WyJzc2gtZWQyNTUxOSBBQUFBQzNOemFDMWxaREkxTlRFNUFBQUFJRmtLdE1mZi9KNEh1N1pORmRMdHMwQmFoaThlRmJ6UExJL283TnN0VDV6dyJdfQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA"), disks: [DiskRequest { device: "nvme", name: "disk-10889371ea3", read_only: false, slot: Slot(1), volume_construction_request: Volume { block_size: 512, id: ffb3dff6-c782-4ce7-800e-13770926bc89, read_only_parent: Some(Url { block_size: 512, id: 4bc1bf58-40f0-46cc-a5ac-f90b12ff5116, url: "http://[fd00:1122:3344:101::1]:54321/debian-11-genericcloud-amd64.raw" }), sub_volumes: [Region { block_size: 512, blocks_per_extent: 131072, extent_count: 32, gen: 1, opts: CrucibleOpts { cert_pem: None, control: None, flush_timeout: None, id: ffb3dff6-c782-4ce7-800e-13770926bc89, key: Some("cKWBBvAjYkTD3jSFxo6Arz79k+dVn6SpvhQyFV1RXnc="), key_pem: None, lossy: false, read_only: false, root_cert_pem: None, target: ["[fd00:1122:3344:101::12]:19000", "[fd00:1122:3344:101::17]:19000", "[fd00:1122:3344:101::19]:19000"] } }] } }], migrate: None, nics: [NetworkInterfaceRequest { name: "vopte6", slot: Slot(0) }], properties: InstanceProperties { bootrom_id: 00000000-0000-0000-0000-000000000000, description: "Test description", id: 798fd184-57e7-4c63-a223-b0a810e00937, image_id: 00000000-0000-0000-0000-000000000000, memory: 1024, name: "localshark", vcpus: 2 } } file = sled-agent/src/instance.rs:450 instance_id = 798fd184-57e7-4c63-a223-b0a810e00937
12522023-12-06T17:43:57.088ZINFOSledAgent (InstanceManager): result of instance_ensure call is Ok(InstanceEnsureResponse { migrate: None }) file = sled-agent/src/instance.rs:452 instance_id = 798fd184-57e7-4c63-a223-b0a810e00937
12532023-12-06T17:43:57.089ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 7459865 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:47331 req_id = 21320fbe-02a2-417a-875d-e2f0527e9059 response_code = 200 uri = /instances/798fd184-57e7-4c63-a223-b0a810e00937/state
12542023-12-06T17:43:57.089ZINFOSledAgent (InstanceManager): Observing new propolis state: ObservedPropolisState { vmm_state: PropolisInstanceState(Starting), migration_status: NoMigration, time: 2023-12-06T17:43:57.089286909Z } file = sled-agent/src/instance.rs:334 instance_id = 798fd184-57e7-4c63-a223-b0a810e00937
12552023-12-06T17:43:57.089ZINFOSledAgent (InstanceManager): updated state after observing Propolis state change file = sled-agent/src/instance.rs:356 instance_id = 798fd184-57e7-4c63-a223-b0a810e00937 new_instance_state = InstanceRuntimeState { propolis_id: Some(ef02d4e7-7031-4005-b456-539acbeb1df5), dst_propolis_id: None, migration_id: None, gen: Generation(3), time_updated: 2023-12-06T17:43:49.039589950Z } new_vmm_state = VmmRuntimeState { state: Starting, gen: Generation(2), time_updated: 2023-12-06T17:43:57.089286909Z } propolis_id = ef02d4e7-7031-4005-b456-539acbeb1df5
12562023-12-06T17:43:57.089ZINFOSledAgent (InstanceManager): Publishing instance state update to Nexus file = sled-agent/src/instance.rs:259 instance_id = 798fd184-57e7-4c63-a223-b0a810e00937 state = SledInstanceState { instance_state: InstanceRuntimeState { propolis_id: Some(ef02d4e7-7031-4005-b456-539acbeb1df5), dst_propolis_id: None, migration_id: None, gen: Generation(3), time_updated: 2023-12-06T17:43:49.039589950Z }, propolis_id: ef02d4e7-7031-4005-b456-539acbeb1df5, vmm_state: VmmRuntimeState { state: Starting, gen: Generation(2), time_updated: 2023-12-06T17:43:57.089286909Z } }
12572023-12-06T17:43:57.142ZINFOSledAgent (InstanceManager): Observing new propolis state: ObservedPropolisState { vmm_state: PropolisInstanceState(Running), migration_status: NoMigration, time: 2023-12-06T17:43:57.142231545Z } file = sled-agent/src/instance.rs:334 instance_id = 798fd184-57e7-4c63-a223-b0a810e00937
12582023-12-06T17:43:57.142ZINFOSledAgent (InstanceManager): updated state after observing Propolis state change file = sled-agent/src/instance.rs:356 instance_id = 798fd184-57e7-4c63-a223-b0a810e00937 new_instance_state = InstanceRuntimeState { propolis_id: Some(ef02d4e7-7031-4005-b456-539acbeb1df5), dst_propolis_id: None, migration_id: None, gen: Generation(3), time_updated: 2023-12-06T17:43:49.039589950Z } new_vmm_state = VmmRuntimeState { state: Running, gen: Generation(3), time_updated: 2023-12-06T17:43:57.142231545Z } propolis_id = ef02d4e7-7031-4005-b456-539acbeb1df5
12592023-12-06T17:43:57.142ZINFOSledAgent (InstanceManager): Publishing instance state update to Nexus file = sled-agent/src/instance.rs:259 instance_id = 798fd184-57e7-4c63-a223-b0a810e00937 state = SledInstanceState { instance_state: InstanceRuntimeState { propolis_id: Some(ef02d4e7-7031-4005-b456-539acbeb1df5), dst_propolis_id: None, migration_id: None, gen: Generation(3), time_updated: 2023-12-06T17:43:49.039589950Z }, propolis_id: ef02d4e7-7031-4005-b456-539acbeb1df5, vmm_state: VmmRuntimeState { state: Running, gen: Generation(3), time_updated: 2023-12-06T17:43:57.142231545Z } }
12602023-12-06T17:43:59.427ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::60fc:8ff:fe95:bcc8": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
12612023-12-06T17:44:04.429ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::60fc:8ff:fe95:bcc8": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
12622023-12-06T17:44:06.387ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:769 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:35198
12632023-12-06T17:44:06.388ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 578 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:35198 req_id = c8c70590-459e-4824-b5d1-475eefe911b4 response_code = 200 uri = /baseboard
12642023-12-06T17:44:09.431ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::60fc:8ff:fe95:bcc8": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
12652023-12-06T17:44:14.432ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::60fc:8ff:fe95:bcc8": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
12662023-12-06T17:44:19.434ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::60fc:8ff:fe95:bcc8": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
12672023-12-06T17:44:19.906ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 228 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::d]:52761 req_id = ce8c04cb-5c1e-4ee2-a62c-ed3651d1d59a response_code = 200 uri = /metrics/collect/1e40a2ae-ffda-4a94-95f6-839f3349172b
12682023-12-06T17:44:24.436ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::60fc:8ff:fe95:bcc8": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
12692023-12-06T17:44:29.438ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::60fc:8ff:fe95:bcc8": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
12702023-12-06T17:44:34.439ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::60fc:8ff:fe95:bcc8": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
12712023-12-06T17:44:36.421ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:769 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:33008
12722023-12-06T17:44:36.421ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 44 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:33008 req_id = 8a09ddfc-4054-455d-a4e7-8359caa85da2 response_code = 200 uri = /baseboard
12732023-12-06T17:44:39.442ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::60fc:8ff:fe95:bcc8": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
12742023-12-06T17:44:44.443ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::60fc:8ff:fe95:bcc8": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
12752023-12-06T17:44:49.444ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::60fc:8ff:fe95:bcc8": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
12762023-12-06T17:44:49.906ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 193 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::d]:52761 req_id = 8a71cc3c-c3de-4d45-be03-c09db6440d26 response_code = 200 uri = /metrics/collect/1e40a2ae-ffda-4a94-95f6-839f3349172b
12772023-12-06T17:44:54.445ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::60fc:8ff:fe95:bcc8": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
12782023-12-06T17:44:59.447ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::60fc:8ff:fe95:bcc8": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
12792023-12-06T17:45:04.449ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::60fc:8ff:fe95:bcc8": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
12802023-12-06T17:45:06.458ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:769 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:46832
12812023-12-06T17:45:06.458ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 505 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:46832 req_id = aca0c52c-daa5-4195-9590-f913ffdb23ff response_code = 200 uri = /baseboard
12822023-12-06T17:45:09.451ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::60fc:8ff:fe95:bcc8": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
12832023-12-06T17:45:14.452ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::60fc:8ff:fe95:bcc8": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
12842023-12-06T17:45:19.453ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::60fc:8ff:fe95:bcc8": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
12852023-12-06T17:45:19.905ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 274 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::d]:52761 req_id = 36b6a2e4-b819-4353-ba5d-13f20e2a02fb response_code = 200 uri = /metrics/collect/1e40a2ae-ffda-4a94-95f6-839f3349172b
12862023-12-06T17:45:24.454ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::60fc:8ff:fe95:bcc8": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
12872023-12-06T17:45:29.454ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::60fc:8ff:fe95:bcc8": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
12882023-12-06T17:45:34.456ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::60fc:8ff:fe95:bcc8": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
12892023-12-06T17:45:36.490ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:769 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:62880
12902023-12-06T17:45:36.490ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 48 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:62880 req_id = 56f0d4f6-88d3-43b1-8542-5996b5d8d8b5 response_code = 200 uri = /baseboard
12912023-12-06T17:45:38.977ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:532
12922023-12-06T17:45:39.458ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::60fc:8ff:fe95:bcc8": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
12932023-12-06T17:45:44.459ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::60fc:8ff:fe95:bcc8": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
12942023-12-06T17:45:49.461ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::60fc:8ff:fe95:bcc8": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
12952023-12-06T17:45:49.906ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 204 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::d]:52761 req_id = c9f8c551-7054-4cfe-b0c1-8f54120e26d1 response_code = 200 uri = /metrics/collect/1e40a2ae-ffda-4a94-95f6-839f3349172b
12962023-12-06T17:45:54.463ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::60fc:8ff:fe95:bcc8": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
12972023-12-06T17:45:59.465ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::60fc:8ff:fe95:bcc8": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
12982023-12-06T17:46:04.466ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::60fc:8ff:fe95:bcc8": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
12992023-12-06T17:46:06.527ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:769 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:51343
13002023-12-06T17:46:06.527ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 61 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:51343 req_id = 6b3d1110-1749-48cf-86e3-fd56cfce1111 response_code = 200 uri = /baseboard
13012023-12-06T17:46:09.468ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::60fc:8ff:fe95:bcc8": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
13022023-12-06T17:46:14.470ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::60fc:8ff:fe95:bcc8": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
13032023-12-06T17:46:19.472ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::60fc:8ff:fe95:bcc8": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
13042023-12-06T17:46:19.906ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 201 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::d]:52761 req_id = 89abc2c0-045f-44c0-9380-743337ff4e6e response_code = 200 uri = /metrics/collect/1e40a2ae-ffda-4a94-95f6-839f3349172b
13052023-12-06T17:46:24.473ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::60fc:8ff:fe95:bcc8": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
13062023-12-06T17:46:29.475ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::60fc:8ff:fe95:bcc8": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
13072023-12-06T17:46:34.476ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::60fc:8ff:fe95:bcc8": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
13082023-12-06T17:46:36.561ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:769 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:48326
13092023-12-06T17:46:36.561ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 469 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:48326 req_id = 334076a0-13c2-4263-bc88-a2ffaf8409e2 response_code = 200 uri = /baseboard
13102023-12-06T17:46:39.076ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:769 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:43866
13112023-12-06T17:46:39.077ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 455 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:43866 req_id = 45203fb3-0717-4101-a76b-dd47424310a7 response_code = 200 uri = /instances/798fd184-57e7-4c63-a223-b0a810e00937/state
13122023-12-06T17:46:39.077ZINFOSledAgent (InstanceManager): Observing new propolis state: ObservedPropolisState { vmm_state: PropolisInstanceState(Stopping), migration_status: NoMigration, time: 2023-12-06T17:46:39.077440127Z } file = sled-agent/src/instance.rs:334 instance_id = 798fd184-57e7-4c63-a223-b0a810e00937
13132023-12-06T17:46:39.077ZINFOSledAgent (InstanceManager): updated state after observing Propolis state change file = sled-agent/src/instance.rs:356 instance_id = 798fd184-57e7-4c63-a223-b0a810e00937 new_instance_state = InstanceRuntimeState { propolis_id: Some(ef02d4e7-7031-4005-b456-539acbeb1df5), dst_propolis_id: None, migration_id: None, gen: Generation(3), time_updated: 2023-12-06T17:43:49.039589950Z } new_vmm_state = VmmRuntimeState { state: Stopping, gen: Generation(5), time_updated: 2023-12-06T17:46:39.077440127Z } propolis_id = ef02d4e7-7031-4005-b456-539acbeb1df5
13142023-12-06T17:46:39.077ZINFOSledAgent (InstanceManager): Publishing instance state update to Nexus file = sled-agent/src/instance.rs:259 instance_id = 798fd184-57e7-4c63-a223-b0a810e00937 state = SledInstanceState { instance_state: InstanceRuntimeState { propolis_id: Some(ef02d4e7-7031-4005-b456-539acbeb1df5), dst_propolis_id: None, migration_id: None, gen: Generation(3), time_updated: 2023-12-06T17:43:49.039589950Z }, propolis_id: ef02d4e7-7031-4005-b456-539acbeb1df5, vmm_state: VmmRuntimeState { state: Stopping, gen: Generation(5), time_updated: 2023-12-06T17:46:39.077440127Z } }
13152023-12-06T17:46:39.100ZINFOSledAgent (InstanceManager): Observing new propolis state: ObservedPropolisState { vmm_state: PropolisInstanceState(Stopped), migration_status: NoMigration, time: 2023-12-06T17:46:39.100886014Z } file = sled-agent/src/instance.rs:334 instance_id = 798fd184-57e7-4c63-a223-b0a810e00937
13162023-12-06T17:46:39.100ZINFOSledAgent (InstanceManager): updated state after observing Propolis state change file = sled-agent/src/instance.rs:356 instance_id = 798fd184-57e7-4c63-a223-b0a810e00937 new_instance_state = InstanceRuntimeState { propolis_id: Some(ef02d4e7-7031-4005-b456-539acbeb1df5), dst_propolis_id: None, migration_id: None, gen: Generation(3), time_updated: 2023-12-06T17:43:49.039589950Z } new_vmm_state = VmmRuntimeState { state: Stopping, gen: Generation(6), time_updated: 2023-12-06T17:46:39.100886014Z } propolis_id = ef02d4e7-7031-4005-b456-539acbeb1df5
13172023-12-06T17:46:39.100ZINFOSledAgent (InstanceManager): Publishing instance state update to Nexus file = sled-agent/src/instance.rs:259 instance_id = 798fd184-57e7-4c63-a223-b0a810e00937 state = SledInstanceState { instance_state: InstanceRuntimeState { propolis_id: Some(ef02d4e7-7031-4005-b456-539acbeb1df5), dst_propolis_id: None, migration_id: None, gen: Generation(3), time_updated: 2023-12-06T17:43:49.039589950Z }, propolis_id: ef02d4e7-7031-4005-b456-539acbeb1df5, vmm_state: VmmRuntimeState { state: Stopping, gen: Generation(6), time_updated: 2023-12-06T17:46:39.100886014Z } }
13182023-12-06T17:46:39.266ZINFOSledAgent (InstanceManager): Observing new propolis state: ObservedPropolisState { vmm_state: PropolisInstanceState(Destroyed), migration_status: NoMigration, time: 2023-12-06T17:46:39.266248931Z } file = sled-agent/src/instance.rs:334 instance_id = 798fd184-57e7-4c63-a223-b0a810e00937
13192023-12-06T17:46:39.266ZINFOSledAgent (InstanceManager): updated state after observing Propolis state change file = sled-agent/src/instance.rs:356 instance_id = 798fd184-57e7-4c63-a223-b0a810e00937 new_instance_state = InstanceRuntimeState { propolis_id: None, dst_propolis_id: None, migration_id: None, gen: Generation(5), time_updated: 2023-12-06T17:46:39.266248931Z } new_vmm_state = VmmRuntimeState { state: Destroyed, gen: Generation(7), time_updated: 2023-12-06T17:46:39.266248931Z } propolis_id = ef02d4e7-7031-4005-b456-539acbeb1df5
13202023-12-06T17:46:39.266ZINFOSledAgent (InstanceManager): terminating VMM that has exited file = sled-agent/src/instance.rs:371 instance_id = 798fd184-57e7-4c63-a223-b0a810e00937
13212023-12-06T17:46:39.266ZINFOSledAgent (StorageManager): Received GetLatestResources(Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) }) file = sled-storage/src/manager.rs:346
13222023-12-06T17:46:39.267ZINFOSledAgent (StorageManager): Received GetLatestResources(Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) }) file = sled-storage/src/manager.rs:346
13232023-12-06T17:46:39.267ZINFOSledAgent (ZoneBundler): creating zone bundle context = ZoneBundleContext { storage_dirs: ["/pool/int/a462a7f7-b628-40fe-80ff-4e4189e2d62b/debug/bundle/zone", "/pool/int/b462a7f7-b628-40fe-80ff-4e4189e2d62b/debug/bundle/zone"], cause: TerminatedInstance, extra_log_dirs: ["/pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/debug", "/pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/debug", "/pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/debug", "/pool/ext/616b26df-e62a-4c68-b506-f4a923d8aaf7/crypt/debug", "/pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/debug", "/pool/ext/ceb4461c-cf56-4719-ad3c-14430bfdfb60/crypt/debug", "/pool/ext/d462a7f7-b628-40fe-80ff-4e4189e2d62b/crypt/debug", "/pool/ext/e4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/debug", "/pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/debug"] } file = sled-agent/src/zone_bundle.rs:436 zone_name = oxz_propolis-server_ef02d4e7-7031-4005-b456-539acbeb1df5
13242023-12-06T17:46:39.478ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::60fc:8ff:fe95:bcc8": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
13252023-12-06T17:46:39.912ZINFOSledAgent (ZoneBundler): finished zone bundle file = sled-agent/src/zone_bundle.rs:1229 metadata = ZoneBundleMetadata { id: ZoneBundleId { zone_name: "oxz_propolis-server_ef02d4e7-7031-4005-b456-539acbeb1df5", bundle_id: fe2afc84-bc9c-4349-a377-309b3a7f8120 }, time_created: 2023-12-06T17:46:39.267226820Z, version: 0, cause: TerminatedInstance }
13262023-12-06T17:46:39.912ZWARNSledAgent (InstanceManager): Halting and removing zone: oxz_propolis-server_ef02d4e7-7031-4005-b456-539acbeb1df5 file = sled-agent/src/instance.rs:544 instance_id = 798fd184-57e7-4c63-a223-b0a810e00937
13272023-12-06T17:46:41.485ZINFOSledAgent (InstanceManager): halt_and_remove_logged: Previous zone state: Running file = illumos-utils/src/zone.rs:268 instance_id = 798fd184-57e7-4c63-a223-b0a810e00937
13282023-12-06T17:46:41.512ZINFOSledAgent (InstanceManager): Publishing instance state update to Nexus file = sled-agent/src/instance.rs:259 instance_id = 798fd184-57e7-4c63-a223-b0a810e00937 state = SledInstanceState { instance_state: InstanceRuntimeState { propolis_id: None, dst_propolis_id: None, migration_id: None, gen: Generation(5), time_updated: 2023-12-06T17:46:39.266248931Z }, propolis_id: ef02d4e7-7031-4005-b456-539acbeb1df5, vmm_state: VmmRuntimeState { state: Destroyed, gen: Generation(7), time_updated: 2023-12-06T17:46:39.266248931Z } }
13292023-12-06T17:46:41.610ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:769 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:36305
13302023-12-06T17:46:41.611ZWARNSledAgent (PortManager): unset_virtual_nic_host unimplmented file = illumos-utils/src/opte/port_manager.rs:494
13312023-12-06T17:46:41.611ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 145 local_addr = [fd00:1122:3344:101::1]:12345 method = DELETE remote_addr = [fd00:1122:3344:101::c]:36305 req_id = 3e352ac5-321f-4c7b-b6dc-b060596c8239 response_code = 204 uri = /v2p/ab76162d-02c5-4805-8660-6d3f946e2148
13322023-12-06T17:46:41.712ZINFOSledAgent (InstanceManager): State monitoring task complete file = sled-agent/src/instance.rs:487 instance_id = 798fd184-57e7-4c63-a223-b0a810e00937