Max level shown:
1 [ Dec 15 01:08:56 Enabled. ]
2 [ Dec 15 01:08:56 Executing start method ("ctrun -l child -o noorphan,regent /opt/oxide/sled-agent/sled-agent run /opt/oxide/sled-agent/pkg/config.toml &"). ]
3 [ Dec 15 01:08:56 Method "start" exited with status 0. ]
4 [ Dec 15 01:08:56 Rereading configuration. ]
5 [ Dec 15 01:08:56 No 'refresh' method defined. Treating as :true. ]
6 note: configured to log to "/dev/stdout"
72023-12-15T01:08:56.473ZINFOSledAgent (BootstrapAgentStartup): finding links ["net0", "net1"] file = sled-agent/src/bootstrap/pre_server.rs:240
82023-12-15T01:08:56.527ZINFOSledAgent (BootstrapAgentStartup): Starting mg-ddm service addr-objs = [AddrObject { interface: "net0", name: "ll" }, AddrObject { interface: "net1", name: "ll" }] file = sled-agent/src/bootstrap/pre_server.rs:247
92023-12-15T01:08:56.527ZINFOSledAgent (BootstrapAgentStartup): Importing mg-ddm service file = sled-agent/src/bootstrap/maghemite.rs:42 path = /opt/oxide/mg-ddm/pkg/ddm/manifest.xml
102023-12-15T01:08:56.639ZINFOSledAgent (BootstrapAgentStartup): Setting mg-ddm interfaces file = sled-agent/src/bootstrap/maghemite.rs:50 interfaces = ("net0/ll" "net1/ll")
112023-12-15T01:08:56.648ZINFOSledAgent (BootstrapAgentStartup): Enabling mg-ddm service file = sled-agent/src/bootstrap/maghemite.rs:56
122023-12-15T01:08:56.656ZINFOSledAgent (BootstrapAgentStartup): Ensuring zfs key directory exists file = sled-agent/src/bootstrap/pre_server.rs:257 path = /var/run/oxide/
132023-12-15T01:08:57.278ZINFOSledAgent (BootstrapAgentStartup): Sending prefix to ddmd for advertisement DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:91 prefix = Ipv6Prefix { addr: fdb0:18c0:4d0d:a02a::, len: 64 }
142023-12-15T01:08:57.414ZINFOSledAgent: Starting KeyManager file = sled-agent/src/long_running_tasks.rs:117
152023-12-15T01:08:57.414ZINFOSledAgent: Starting StorageManager file = sled-agent/src/long_running_tasks.rs:129
162023-12-15T01:08:57.414ZINFOSledAgent: Starting StorageMonitor file = sled-agent/src/long_running_tasks.rs:141
172023-12-15T01:08:57.414ZINFOSledAgent: Starting HardwareManager file = sled-agent/src/long_running_tasks.rs:162 sled_mode = Scrimlet { asic: SoftNpuZone }
182023-12-15T01:08:57.414ZINFOSledAgent (DumpSetup-archival): DumpSetup poll loop started. file = sled-agent/src/dump_setup.rs:171
192023-12-15T01:08:57.414ZINFOSledAgent (HardwareManager): Creating HardwareManager file = sled-hardware/src/illumos/mod.rs:606
202023-12-15T01:08:57.414ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:532
212023-12-15T01:08:57.417ZINFOSledAgent (HardwareManager): Generated i86pc baseboard Pc { identifier: "37fc7d027c1a4e9eb4f7b462ba03b0a1", model: "i86pc" } file = sled-hardware/src/illumos/mod.rs:532
222023-12-15T01:08:57.417ZWARNSledAgent (HardwareManager): Device is not a Gimlet (i86pc), proceeding with null hardware view file = sled-hardware/src/illumos/mod.rs:655
232023-12-15T01:08:57.417ZINFOSledAgent: Starting HardwareMonitor file = sled-agent/src/long_running_tasks.rs:176
242023-12-15T01:08:57.417ZINFOSledAgent (HardwareManager): Monitoring for hardware updates file = sled-hardware/src/illumos/mod.rs:713
252023-12-15T01:08:57.417ZINFOSledAgent: Upserting synthetic zpool to Storage Manager: oxi_a462a7f7-b628-40fe-80ff-4e4189e2d62b file = sled-agent/src/long_running_tasks.rs:232
262023-12-15T01:08:57.417ZINFOSledAgent: Upserting synthetic zpool to Storage Manager: oxi_b462a7f7-b628-40fe-80ff-4e4189e2d62b file = sled-agent/src/long_running_tasks.rs:232
272023-12-15T01:08:57.417ZINFOSledAgent: Upserting synthetic zpool to Storage Manager: oxp_d462a7f7-b628-40fe-80ff-4e4189e2d62b file = sled-agent/src/long_running_tasks.rs:232
282023-12-15T01:08:57.417ZINFOSledAgent: Upserting synthetic zpool to Storage Manager: oxp_e4b4dc87-ab46-49fb-a4b4-d361ae214c03 file = sled-agent/src/long_running_tasks.rs:232
292023-12-15T01:08:57.417ZINFOSledAgent: Upserting synthetic zpool to Storage Manager: oxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03 file = sled-agent/src/long_running_tasks.rs:232
302023-12-15T01:08:57.417ZINFOSledAgent: Upserting synthetic zpool to Storage Manager: oxp_14b4dc87-ab46-49fb-a4b4-d361ae214c03 file = sled-agent/src/long_running_tasks.rs:232
312023-12-15T01:08:57.417ZINFOSledAgent: Upserting synthetic zpool to Storage Manager: oxp_24b4dc87-ab46-49fb-a4b4-d361ae214c03 file = sled-agent/src/long_running_tasks.rs:232
322023-12-15T01:08:57.417ZINFOSledAgent: Upserting synthetic zpool to Storage Manager: oxp_cd70d7f6-2354-4bf2-8012-55bf9eaf7930 file = sled-agent/src/long_running_tasks.rs:232
332023-12-15T01:08:57.417ZINFOSledAgent: Upserting synthetic zpool to Storage Manager: oxp_ceb4461c-cf56-4719-ad3c-14430bfdfb60 file = sled-agent/src/long_running_tasks.rs:232
342023-12-15T01:08:57.417ZINFOSledAgent: Upserting synthetic zpool to Storage Manager: oxp_31bd71cd-4736-4a12-a387-9b74b050396f file = sled-agent/src/long_running_tasks.rs:232
352023-12-15T01:08:57.417ZINFOSledAgent: Upserting synthetic zpool to Storage Manager: oxp_616b26df-e62a-4c68-b506-f4a923d8aaf7 file = sled-agent/src/long_running_tasks.rs:232
362023-12-15T01:08:57.417ZINFOSledAgent: Waiting for boot disk file = sled-agent/src/long_running_tasks.rs:88
372023-12-15T01:08:57.417ZINFOSledAgent (HardwareMonitor): Checking current full hardware snapshot file = sled-agent/src/hardware_monitor.rs:241 underlay_network_info = None
382023-12-15T01:08:57.417ZINFOSledAgent (StorageManager): Received AddDisk(Synthetic(SyntheticDisk { identity: DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-a462a7f7-b628-40fe-80ff-4e4189e2d62b", model: "synthetic-model" }, zpool_name: ZpoolName { id: a462a7f7-b628-40fe-80ff-4e4189e2d62b, kind: Internal } })) file = sled-storage/src/manager.rs:346
392023-12-15T01:08:57.604ZINFOSledAgent (StorageManager): Channel capacity decreased current = 245 file = sled-storage/src/manager.rs:326 previous = 256
402023-12-15T01:08:57.606ZINFOSledAgent: Found boot disk DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-a462a7f7-b628-40fe-80ff-4e4189e2d62b", model: "synthetic-model" } file = sled-agent/src/long_running_tasks.rs:90
412023-12-15T01:08:57.607ZINFOSledAgent (StorageManager): Received AddDisk(Synthetic(SyntheticDisk { identity: DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-b462a7f7-b628-40fe-80ff-4e4189e2d62b", model: "synthetic-model" }, zpool_name: ZpoolName { id: b462a7f7-b628-40fe-80ff-4e4189e2d62b, kind: Internal } })) file = sled-storage/src/manager.rs:346
422023-12-15T01:08:57.782ZINFOSledAgent (StorageManager): Received AddDisk(Synthetic(SyntheticDisk { identity: DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-d462a7f7-b628-40fe-80ff-4e4189e2d62b", model: "synthetic-model" }, zpool_name: ZpoolName { id: d462a7f7-b628-40fe-80ff-4e4189e2d62b, kind: External } })) file = sled-storage/src/manager.rs:346
432023-12-15T01:08:57.782ZINFOSledAgent (StorageManager): Received AddDisk(Synthetic(SyntheticDisk { identity: DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-e4b4dc87-ab46-49fb-a4b4-d361ae214c03", model: "synthetic-model" }, zpool_name: ZpoolName { id: e4b4dc87-ab46-49fb-a4b4-d361ae214c03, kind: External } })) file = sled-storage/src/manager.rs:346
442023-12-15T01:08:57.782ZINFOSledAgent (StorageManager): Received AddDisk(Synthetic(SyntheticDisk { identity: DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-f4b4dc87-ab46-49fb-a4b4-d361ae214c03", model: "synthetic-model" }, zpool_name: ZpoolName { id: f4b4dc87-ab46-49fb-a4b4-d361ae214c03, kind: External } })) file = sled-storage/src/manager.rs:346
452023-12-15T01:08:57.782ZINFOSledAgent (StorageManager): Received AddDisk(Synthetic(SyntheticDisk { identity: DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-14b4dc87-ab46-49fb-a4b4-d361ae214c03", model: "synthetic-model" }, zpool_name: ZpoolName { id: 14b4dc87-ab46-49fb-a4b4-d361ae214c03, kind: External } })) file = sled-storage/src/manager.rs:346
462023-12-15T01:08:57.782ZINFOSledAgent (StorageManager): Received AddDisk(Synthetic(SyntheticDisk { identity: DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-24b4dc87-ab46-49fb-a4b4-d361ae214c03", model: "synthetic-model" }, zpool_name: ZpoolName { id: 24b4dc87-ab46-49fb-a4b4-d361ae214c03, kind: External } })) file = sled-storage/src/manager.rs:346
472023-12-15T01:08:57.782ZINFOSledAgent (StorageManager): Received AddDisk(Synthetic(SyntheticDisk { identity: DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-cd70d7f6-2354-4bf2-8012-55bf9eaf7930", model: "synthetic-model" }, zpool_name: ZpoolName { id: cd70d7f6-2354-4bf2-8012-55bf9eaf7930, kind: External } })) file = sled-storage/src/manager.rs:346
482023-12-15T01:08:57.782ZINFOSledAgent (StorageManager): Received AddDisk(Synthetic(SyntheticDisk { identity: DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-ceb4461c-cf56-4719-ad3c-14430bfdfb60", model: "synthetic-model" }, zpool_name: ZpoolName { id: ceb4461c-cf56-4719-ad3c-14430bfdfb60, kind: External } })) file = sled-storage/src/manager.rs:346
492023-12-15T01:08:57.782ZINFOSledAgent (StorageManager): Received AddDisk(Synthetic(SyntheticDisk { identity: DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-31bd71cd-4736-4a12-a387-9b74b050396f", model: "synthetic-model" }, zpool_name: ZpoolName { id: 31bd71cd-4736-4a12-a387-9b74b050396f, kind: External } })) file = sled-storage/src/manager.rs:346
502023-12-15T01:08:57.782ZINFOSledAgent (StorageManager): Received AddDisk(Synthetic(SyntheticDisk { identity: DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-616b26df-e62a-4c68-b506-f4a923d8aaf7", model: "synthetic-model" }, zpool_name: ZpoolName { id: 616b26df-e62a-4c68-b506-f4a923d8aaf7, kind: External } })) file = sled-storage/src/manager.rs:346
512023-12-15T01:08:57.782ZINFOSledAgent (StorageManager): Received DisksChanged({}) file = sled-storage/src/manager.rs:346
522023-12-15T01:08:57.782ZINFOSledAgent (StorageManager): Channel capacity increased current = 255 file = sled-storage/src/manager.rs:336 previous = 245
532023-12-15T01:08:57.782ZINFOSledAgent (StorageManager): Received GetLatestResources(Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) }) file = sled-storage/src/manager.rs:346
542023-12-15T01:08:57.782ZINFOSledAgent (StorageMonitor): Received storage manager update file = sled-agent/src/storage_monitor.rs:111 resources = StorageResources { disks: {DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-a462a7f7-b628-40fe-80ff-4e4189e2d62b", model: "synthetic-model" }: (Synthetic(SyntheticDisk { identity: DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-a462a7f7-b628-40fe-80ff-4e4189e2d62b", model: "synthetic-model" }, zpool_name: ZpoolName { id: a462a7f7-b628-40fe-80ff-4e4189e2d62b, kind: Internal } }), Pool { name: ZpoolName { id: a462a7f7-b628-40fe-80ff-4e4189e2d62b, kind: Internal }, info: ZpoolInfo { name: "oxi_a462a7f7-b628-40fe-80ff-4e4189e2d62b", size: 20937965568, allocated: 542208, free: 20937423360, health: Online }, parent: DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-a462a7f7-b628-40fe-80ff-4e4189e2d62b", model: "synthetic-model" } }), DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-b462a7f7-b628-40fe-80ff-4e4189e2d62b", model: "synthetic-model" }: (Synthetic(SyntheticDisk { identity: DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-b462a7f7-b628-40fe-80ff-4e4189e2d62b", model: "synthetic-model" }, zpool_name: ZpoolName { id: b462a7f7-b628-40fe-80ff-4e4189e2d62b, kind: Internal } }), Pool { name: ZpoolName { id: b462a7f7-b628-40fe-80ff-4e4189e2d62b, kind: Internal }, info: ZpoolInfo { name: "oxi_b462a7f7-b628-40fe-80ff-4e4189e2d62b", size: 20937965568, allocated: 493056, free: 20937472512, health: Online }, parent: DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-b462a7f7-b628-40fe-80ff-4e4189e2d62b", model: "synthetic-model" } })} }
552023-12-15T01:08:57.782ZINFOSledAgent: Starting Bootstore file = sled-agent/src/long_running_tasks.rs:200
562023-12-15T01:08:57.782ZINFOSledAgent (bootstore): No ledger in /pool/int/a462a7f7-b628-40fe-80ff-4e4189e2d62b/cluster/bootstore-fsm-state.json file = common/src/ledger.rs:187 peer_id = pc-37fc7d027c1a4e9eb4f7b462ba03b0a1-i86pc
572023-12-15T01:08:57.782ZINFOSledAgent (bootstore): No ledger in /pool/int/b462a7f7-b628-40fe-80ff-4e4189e2d62b/cluster/bootstore-fsm-state.json file = common/src/ledger.rs:187 peer_id = pc-37fc7d027c1a4e9eb4f7b462ba03b0a1-i86pc
582023-12-15T01:08:57.782ZINFOSledAgent (bootstore): No ledger found. Loading Fsm::State as Uninitialized file = bootstore/src/schemes/v0/storage.rs:83 peer_id = pc-37fc7d027c1a4e9eb4f7b462ba03b0a1-i86pc
592023-12-15T01:08:57.782ZINFOSledAgent (bootstore): No ledger in /pool/int/a462a7f7-b628-40fe-80ff-4e4189e2d62b/cluster/bootstore-network-config.json file = common/src/ledger.rs:187 peer_id = pc-37fc7d027c1a4e9eb4f7b462ba03b0a1-i86pc
602023-12-15T01:08:57.782ZINFOSledAgent (bootstore): No ledger in /pool/int/b462a7f7-b628-40fe-80ff-4e4189e2d62b/cluster/bootstore-network-config.json file = common/src/ledger.rs:187 peer_id = pc-37fc7d027c1a4e9eb4f7b462ba03b0a1-i86pc
612023-12-15T01:08:57.782ZINFOSledAgent (bootstore): No ledger found for network config file = bootstore/src/schemes/v0/storage.rs:146 peer_id = pc-37fc7d027c1a4e9eb4f7b462ba03b0a1-i86pc
622023-12-15T01:08:57.782ZINFOSledAgent: Starting Bootstore DDMD poller file = sled-agent/src/long_running_tasks.rs:205
632023-12-15T01:08:57.782ZINFOSledAgent: Starting ZoneBundler related tasks file = sled-agent/src/long_running_tasks.rs:220
642023-12-15T01:08:57.796ZINFOSledAgent (ServiceManager): Creating ServiceManager file = sled-agent/src/services.rs:575
652023-12-15T01:08:57.796ZINFOSledAgent (auto-cleanup-task): top of bundle cleanup loop file = sled-agent/src/zone_bundle.rs:288 next_cleanup = Instant { tv_sec: 1610, tv_nsec: 126703728 } time_to_next_cleanup = 599.9999608s
662023-12-15T01:08:57.796ZINFOSledAgent (ServiceManager): Ensuring scrimlet services (enabling services) file = sled-agent/src/services.rs:3037
672023-12-15T01:08:57.796ZINFOSledAgent (StorageManager): Received GetLatestResources(Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) }) file = sled-storage/src/manager.rs:346
682023-12-15T01:08:57.799ZINFOSledAgent (BootstrapAgentStartup): No ledger in /pool/int/a462a7f7-b628-40fe-80ff-4e4189e2d62b/config/sled-agent-request.json file = common/src/ledger.rs:187
692023-12-15T01:08:57.799ZINFOSledAgent (BootstrapAgentStartup): No ledger in /pool/int/b462a7f7-b628-40fe-80ff-4e4189e2d62b/config/sled-agent-request.json file = common/src/ledger.rs:187
702023-12-15T01:08:57.799ZINFOSledAgent (dropshot (BootstrapAgent)): listening file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:195 local_addr = [fdb0:18c0:4d0d:a02a::1]:80
712023-12-15T01:08:57.799ZINFOSledAgent (SledAgentSprocketsServer): Started listening file = sled-agent/src/bootstrap/sprockets_server.rs:44 local_addr = [fdb0:18c0:4d0d:a02a::1]:12346
722023-12-15T01:08:57.800ZINFOSledAgent (RSS): Injecting RSS configuration: RackInitializeRequest { rack_subnet: fd00:1122:3344:100::, trust_quorum_peers: None, bootstrap_discovery: OnlyOurs, ntp_servers: [ "0.pool.ntp.org", ], dns_servers: [ 1.1.1.1, 9.9.9.9, ], internal_services_ip_pool_ranges: [ V4( Ipv4Range { first: 10.151.2.164, last: 10.151.2.173, }, ), ], external_dns_ips: [ 10.151.2.164, 10.151.2.165, ], external_dns_zone_name: "oxide.test", external_certificates: "<redacted>", recovery_silo: RecoverySiloConfig { silo_name: Name( "recovery", ), user_name: UserId( "recovery", ), user_password_hash: NewPasswordHash( PasswordHashString { string: "$argon2id$v=19$m=98304,t=13,p=1$RUlWc0ZxaHo0WFdrN0N6ZQ$S8p52j85GPvMhR/ek3GL0el/oProgTwWpHJZ8lsQQoY", encoding: B64, }, ), }, rack_network_config: Some( RackNetworkConfigV1 { rack_subnet: Ipv6Network { addr: fd00:1122:3344:1::, prefix: 56, }, infra_ip_first: 10.151.2.174, infra_ip_last: 10.151.2.174, ports: [ PortConfigV1 { routes: [ RouteConfig { destination: V4( Ipv4Network { addr: 0.0.0.0, prefix: 0, }, ), nexthop: 10.151.2.100, }, ], addresses: [ V4( Ipv4Network { addr: 10.151.2.174, prefix: 32, }, ), ], switch: Switch0, port: "qsfp0", uplink_port_speed: Speed40G, uplink_port_fec: None, bgp_peers: [], autoneg: false, }, ], bgp: [], }, ), } file = sled-agent/src/rack_setup/service.rs:803
732023-12-15T01:08:57.800ZINFOSledAgent (DnsResolver): new DNS resolver addresses = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53, [fd00:1122:3344:4::1]:53, [fd00:1122:3344:5::1]:53] file = internal-dns/src/resolver.rs:60
742023-12-15T01:08:57.800ZINFOSledAgent (StorageManager): Received GetLatestResources(Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) }) file = sled-storage/src/manager.rs:346
752023-12-15T01:08:57.800ZINFOSledAgent (RSS): No ledger in /pool/int/a462a7f7-b628-40fe-80ff-4e4189e2d62b/config/rss-plan-completed.marker file = common/src/ledger.rs:187
762023-12-15T01:08:57.800ZINFOSledAgent (RSS): No ledger in /pool/int/b462a7f7-b628-40fe-80ff-4e4189e2d62b/config/rss-plan-completed.marker file = common/src/ledger.rs:187
772023-12-15T01:08:57.800ZINFOSledAgent (RSS): RSS configuration has not been fully applied yet file = sled-agent/src/rack_setup/service.rs:863
782023-12-15T01:08:57.800ZINFOSledAgent (StorageManager): Received GetLatestResources(Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) }) file = sled-storage/src/manager.rs:346
792023-12-15T01:08:57.800ZINFOSledAgent (RSS): No ledger in /pool/int/a462a7f7-b628-40fe-80ff-4e4189e2d62b/config/rss-sled-plan.json file = common/src/ledger.rs:187
802023-12-15T01:08:57.800ZINFOSledAgent (RSS): No ledger in /pool/int/b462a7f7-b628-40fe-80ff-4e4189e2d62b/config/rss-sled-plan.json file = common/src/ledger.rs:187
812023-12-15T01:08:57.800ZINFOSledAgent (RSS): Creating new allocation plan file = sled-agent/src/rack_setup/service.rs:904
822023-12-15T01:08:57.800ZINFOSledAgent (RSS): Serializing plan file = sled-agent/src/rack_setup/plan/sled.rs:118
832023-12-15T01:08:57.800ZINFOSledAgent (RSS): Creating plan for the sled at fdb0:18c0:4d0d:a02a::1 file = sled-agent/src/rack_setup/plan/sled.rs:91
842023-12-15T01:08:57.800ZINFOSledAgent (StorageManager): Received GetLatestResources(Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) }) file = sled-storage/src/manager.rs:346
852023-12-15T01:08:57.800ZINFOSledAgent (RSS): Writing ledger to /pool/int/a462a7f7-b628-40fe-80ff-4e4189e2d62b/config/.rss-sled-plan.json.tmp file = common/src/ledger.rs:198
862023-12-15T01:08:57.800ZINFOSledAgent (RSS): Writing ledger to /pool/int/b462a7f7-b628-40fe-80ff-4e4189e2d62b/config/.rss-sled-plan.json.tmp file = common/src/ledger.rs:198
872023-12-15T01:08:57.800ZINFOSledAgent (RSS): Sled plan written to storage file = sled-agent/src/rack_setup/plan/sled.rs:138
882023-12-15T01:08:57.800ZINFOSledAgent (RSS): Writing Rack Network Configuration to bootstore file = sled-agent/src/rack_setup/service.rs:937
892023-12-15T01:08:57.801ZINFOSledAgent (bootstore): Attempting to update network config with generation: 1, current_generation: 0 file = bootstore/src/schemes/v0/peer.rs:556 peer_id = pc-37fc7d027c1a4e9eb4f7b462ba03b0a1-i86pc
902023-12-15T01:08:57.801ZINFOSledAgent (bootstore): Writing ledger to /pool/int/a462a7f7-b628-40fe-80ff-4e4189e2d62b/cluster/.bootstore-network-config.json.tmp file = common/src/ledger.rs:198 peer_id = pc-37fc7d027c1a4e9eb4f7b462ba03b0a1-i86pc
912023-12-15T01:08:57.801ZINFOSledAgent (bootstore): Writing ledger to /pool/int/b462a7f7-b628-40fe-80ff-4e4189e2d62b/cluster/.bootstore-network-config.json.tmp file = common/src/ledger.rs:198 peer_id = pc-37fc7d027c1a4e9eb4f7b462ba03b0a1-i86pc
922023-12-15T01:08:57.801ZINFOSledAgent (bootstore): Broadcasting network config with generation 1 file = bootstore/src/schemes/v0/peer.rs:640 peer_id = pc-37fc7d027c1a4e9eb4f7b462ba03b0a1-i86pc
932023-12-15T01:08:57.801ZINFOSledAgent (BootstrapAgentRssHandler): Received initialization request from RSS file = sled-agent/src/bootstrap/rss_handle.rs:217 request = StartSledAgentRequest { generation: 0, schema_version: 1, body: StartSledAgentRequestBody { id: 78829061-d92a-47ac-bcd6-186a0b9f0582, rack_id: 5628bca7-ce21-4723-b380-345b839f80b4, use_trust_quorum: false, is_lrtq_learner: false, subnet: Ipv6Subnet { net: Ipv6Net(Ipv6Network { addr: fd00:1122:3344:101::, prefix: 64 }) } } } target_sled = [fdb0:18c0:4d0d:a02a::1]:12346
942023-12-15T01:08:57.801ZINFOSledAgent (SledAgentSprocketsServer): Accepted connection file = sled-agent/src/bootstrap/sprockets_server.rs:66 remote_addr = [fdb0:18c0:4d0d:a02a::1]:52055
952023-12-15T01:08:57.801ZINFOSledAgent (SledAgentMain): Loading Sled Agent: StartSledAgentRequest { generation: 0, schema_version: 1, body: StartSledAgentRequestBody { id: 78829061-d92a-47ac-bcd6-186a0b9f0582, rack_id: 5628bca7-ce21-4723-b380-345b839f80b4, use_trust_quorum: false, is_lrtq_learner: false, subnet: Ipv6Subnet { net: Ipv6Net(Ipv6Network { addr: fd00:1122:3344:101::, prefix: 64 }) } } } file = sled-agent/src/bootstrap/server.rs:364
962023-12-15T01:08:57.801ZINFOSledAgent (SledAgentMain): KeyManager: using hardcoded secret retriever file = sled-agent/src/bootstrap/server.rs:379
972023-12-15T01:08:57.801ZINFOSledAgent: setting up sled agent server file = sled-agent/src/server.rs:47
982023-12-15T01:08:57.801ZINFOSledAgent (StorageManager): Received KeyManagerReady file = sled-storage/src/manager.rs:346
992023-12-15T01:08:57.801ZINFOSledAgent (StorageManager): Attempting to add queued disks file = sled-storage/src/manager.rs:394 num_disks = 9
1002023-12-15T01:08:57.801ZINFOSledAgent (DnsResolver): new DNS resolver addresses = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53, [fd00:1122:3344:4::1]:53, [fd00:1122:3344:5::1]:53] file = internal-dns/src/resolver.rs:60
1012023-12-15T01:08:57.804ZINFOSledAgent (ServiceManager): Enabling switch zone (new) file = sled-agent/src/services.rs:3265
1022023-12-15T01:08:57.810ZINFOSledAgent (StorageManager): Loading latest secret disk_id = DiskIdentity {\n vendor: "synthetic-vendor",\n serial: "synthetic-serial-f4b4dc87-ab46-49fb-a4b4-d361ae214c03",\n model: "synthetic-model",\n} file = sled-storage/src/dataset.rs:261
1032023-12-15T01:08:57.810ZINFOSledAgent (StorageManager): Loaded latest secret disk_id = DiskIdentity {\n vendor: "synthetic-vendor",\n serial: "synthetic-serial-f4b4dc87-ab46-49fb-a4b4-d361ae214c03",\n model: "synthetic-model",\n} epoch = 0 file = sled-storage/src/dataset.rs:263
1042023-12-15T01:08:57.810ZINFOSledAgent (StorageManager): Retrieving key disk_id = DiskIdentity {\n vendor: "synthetic-vendor",\n serial: "synthetic-serial-f4b4dc87-ab46-49fb-a4b4-d361ae214c03",\n model: "synthetic-model",\n} epoch = 0 file = sled-storage/src/dataset.rs:267
1052023-12-15T01:08:57.810ZINFOSledAgent (StorageManager): Got key disk_id = DiskIdentity {\n vendor: "synthetic-vendor",\n serial: "synthetic-serial-f4b4dc87-ab46-49fb-a4b4-d361ae214c03",\n model: "synthetic-model",\n} epoch = 0 file = sled-storage/src/dataset.rs:269
1062023-12-15T01:08:57.810ZINFOSledAgent (StorageManager): Created keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-f4b4dc87-ab46-49fb-a4b4-d361ae214c03-synthetic-model-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:37
1072023-12-15T01:08:57.810ZINFOSledAgent (StorageManager): Ensuring encrypted filesystem: crypt for epoch 0 file = sled-storage/src/dataset.rs:281
1082023-12-15T01:08:57.841ZINFOSledAgent (StorageManager): Zeroed and unlinked keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-f4b4dc87-ab46-49fb-a4b4-d361ae214c03-synthetic-model-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:56
1092023-12-15T01:08:57.848ZINFOSledAgent (StorageManager): Automatically destroying dataset: oxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone file = sled-storage/src/dataset.rs:323
1102023-12-15T01:08:57.867ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {}
1112023-12-15T01:08:57.880ZINFOSledAgent: SledAgent::new(..) starting file = sled-agent/src/sled_agent.rs:309 sled_id = 78829061-d92a-47ac-bcd6-186a0b9f0582
1122023-12-15T01:08:57.880ZINFOSledAgent (BootstrapAgentStartup): Sending prefix to ddmd for advertisement DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:91 prefix = Ipv6Prefix { addr: fd00:1122:3344:101::, len: 64 }
1132023-12-15T01:08:57.960ZINFOSledAgent (StorageManager): Loading latest secret disk_id = DiskIdentity {\n vendor: "synthetic-vendor",\n serial: "synthetic-serial-616b26df-e62a-4c68-b506-f4a923d8aaf7",\n model: "synthetic-model",\n} file = sled-storage/src/dataset.rs:261
1142023-12-15T01:08:57.960ZINFOSledAgent (StorageManager): Loaded latest secret disk_id = DiskIdentity {\n vendor: "synthetic-vendor",\n serial: "synthetic-serial-616b26df-e62a-4c68-b506-f4a923d8aaf7",\n model: "synthetic-model",\n} epoch = 0 file = sled-storage/src/dataset.rs:263
1152023-12-15T01:08:57.960ZINFOSledAgent (StorageManager): Retrieving key disk_id = DiskIdentity {\n vendor: "synthetic-vendor",\n serial: "synthetic-serial-616b26df-e62a-4c68-b506-f4a923d8aaf7",\n model: "synthetic-model",\n} epoch = 0 file = sled-storage/src/dataset.rs:267
1162023-12-15T01:08:57.960ZINFOSledAgent (StorageManager): Got key disk_id = DiskIdentity {\n vendor: "synthetic-vendor",\n serial: "synthetic-serial-616b26df-e62a-4c68-b506-f4a923d8aaf7",\n model: "synthetic-model",\n} epoch = 0 file = sled-storage/src/dataset.rs:269
1172023-12-15T01:08:57.960ZINFOSledAgent (StorageManager): Created keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-616b26df-e62a-4c68-b506-f4a923d8aaf7-synthetic-model-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:37
1182023-12-15T01:08:57.960ZINFOSledAgent (StorageManager): Ensuring encrypted filesystem: crypt for epoch 0 file = sled-storage/src/dataset.rs:281
1192023-12-15T01:08:57.987ZINFOSledAgent (StorageManager): Zeroed and unlinked keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-616b26df-e62a-4c68-b506-f4a923d8aaf7-synthetic-model-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:56
1202023-12-15T01:08:57.994ZINFOSledAgent (StorageManager): Automatically destroying dataset: oxp_616b26df-e62a-4c68-b506-f4a923d8aaf7/crypt/zone file = sled-storage/src/dataset.rs:323
1212023-12-15T01:08:58.111ZINFOSledAgent (StorageManager): Loading latest secret disk_id = DiskIdentity {\n vendor: "synthetic-vendor",\n serial: "synthetic-serial-d462a7f7-b628-40fe-80ff-4e4189e2d62b",\n model: "synthetic-model",\n} file = sled-storage/src/dataset.rs:261
1222023-12-15T01:08:58.111ZINFOSledAgent (StorageManager): Loaded latest secret disk_id = DiskIdentity {\n vendor: "synthetic-vendor",\n serial: "synthetic-serial-d462a7f7-b628-40fe-80ff-4e4189e2d62b",\n model: "synthetic-model",\n} epoch = 0 file = sled-storage/src/dataset.rs:263
1232023-12-15T01:08:58.111ZINFOSledAgent (StorageManager): Retrieving key disk_id = DiskIdentity {\n vendor: "synthetic-vendor",\n serial: "synthetic-serial-d462a7f7-b628-40fe-80ff-4e4189e2d62b",\n model: "synthetic-model",\n} epoch = 0 file = sled-storage/src/dataset.rs:267
1242023-12-15T01:08:58.111ZINFOSledAgent (StorageManager): Got key disk_id = DiskIdentity {\n vendor: "synthetic-vendor",\n serial: "synthetic-serial-d462a7f7-b628-40fe-80ff-4e4189e2d62b",\n model: "synthetic-model",\n} epoch = 0 file = sled-storage/src/dataset.rs:269
1252023-12-15T01:08:58.111ZINFOSledAgent (StorageManager): Created keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-d462a7f7-b628-40fe-80ff-4e4189e2d62b-synthetic-model-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:37
1262023-12-15T01:08:58.111ZINFOSledAgent (StorageManager): Ensuring encrypted filesystem: crypt for epoch 0 file = sled-storage/src/dataset.rs:281
1272023-12-15T01:08:58.137ZINFOSledAgent (StorageManager): Zeroed and unlinked keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-d462a7f7-b628-40fe-80ff-4e4189e2d62b-synthetic-model-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:56
1282023-12-15T01:08:58.145ZINFOSledAgent (StorageManager): Automatically destroying dataset: oxp_d462a7f7-b628-40fe-80ff-4e4189e2d62b/crypt/zone file = sled-storage/src/dataset.rs:323
1292023-12-15T01:08:58.253ZINFOSledAgent (StorageManager): Loading latest secret disk_id = DiskIdentity {\n vendor: "synthetic-vendor",\n serial: "synthetic-serial-cd70d7f6-2354-4bf2-8012-55bf9eaf7930",\n model: "synthetic-model",\n} file = sled-storage/src/dataset.rs:261
1302023-12-15T01:08:58.253ZINFOSledAgent (StorageManager): Loaded latest secret disk_id = DiskIdentity {\n vendor: "synthetic-vendor",\n serial: "synthetic-serial-cd70d7f6-2354-4bf2-8012-55bf9eaf7930",\n model: "synthetic-model",\n} epoch = 0 file = sled-storage/src/dataset.rs:263
1312023-12-15T01:08:58.253ZINFOSledAgent (StorageManager): Retrieving key disk_id = DiskIdentity {\n vendor: "synthetic-vendor",\n serial: "synthetic-serial-cd70d7f6-2354-4bf2-8012-55bf9eaf7930",\n model: "synthetic-model",\n} epoch = 0 file = sled-storage/src/dataset.rs:267
1322023-12-15T01:08:58.253ZINFOSledAgent (StorageManager): Got key disk_id = DiskIdentity {\n vendor: "synthetic-vendor",\n serial: "synthetic-serial-cd70d7f6-2354-4bf2-8012-55bf9eaf7930",\n model: "synthetic-model",\n} epoch = 0 file = sled-storage/src/dataset.rs:269
1332023-12-15T01:08:58.253ZINFOSledAgent (StorageManager): Created keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-cd70d7f6-2354-4bf2-8012-55bf9eaf7930-synthetic-model-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:37
1342023-12-15T01:08:58.253ZINFOSledAgent (StorageManager): Ensuring encrypted filesystem: crypt for epoch 0 file = sled-storage/src/dataset.rs:281
1352023-12-15T01:08:58.280ZINFOSledAgent (StorageManager): Zeroed and unlinked keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-cd70d7f6-2354-4bf2-8012-55bf9eaf7930-synthetic-model-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:56
1362023-12-15T01:08:58.287ZINFOSledAgent (StorageManager): Automatically destroying dataset: oxp_cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone file = sled-storage/src/dataset.rs:323
1372023-12-15T01:08:58.399ZINFOSledAgent (StorageManager): Loading latest secret disk_id = DiskIdentity {\n vendor: "synthetic-vendor",\n serial: "synthetic-serial-ceb4461c-cf56-4719-ad3c-14430bfdfb60",\n model: "synthetic-model",\n} file = sled-storage/src/dataset.rs:261
1382023-12-15T01:08:58.399ZINFOSledAgent (StorageManager): Loaded latest secret disk_id = DiskIdentity {\n vendor: "synthetic-vendor",\n serial: "synthetic-serial-ceb4461c-cf56-4719-ad3c-14430bfdfb60",\n model: "synthetic-model",\n} epoch = 0 file = sled-storage/src/dataset.rs:263
1392023-12-15T01:08:58.399ZINFOSledAgent (StorageManager): Retrieving key disk_id = DiskIdentity {\n vendor: "synthetic-vendor",\n serial: "synthetic-serial-ceb4461c-cf56-4719-ad3c-14430bfdfb60",\n model: "synthetic-model",\n} epoch = 0 file = sled-storage/src/dataset.rs:267
1402023-12-15T01:08:58.399ZINFOSledAgent (StorageManager): Got key disk_id = DiskIdentity {\n vendor: "synthetic-vendor",\n serial: "synthetic-serial-ceb4461c-cf56-4719-ad3c-14430bfdfb60",\n model: "synthetic-model",\n} epoch = 0 file = sled-storage/src/dataset.rs:269
1412023-12-15T01:08:58.399ZINFOSledAgent (StorageManager): Created keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-ceb4461c-cf56-4719-ad3c-14430bfdfb60-synthetic-model-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:37
1422023-12-15T01:08:58.399ZINFOSledAgent (StorageManager): Ensuring encrypted filesystem: crypt for epoch 0 file = sled-storage/src/dataset.rs:281
1432023-12-15T01:08:58.426ZINFOSledAgent (StorageManager): Zeroed and unlinked keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-ceb4461c-cf56-4719-ad3c-14430bfdfb60-synthetic-model-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:56
1442023-12-15T01:08:58.435ZINFOSledAgent (StorageManager): Automatically destroying dataset: oxp_ceb4461c-cf56-4719-ad3c-14430bfdfb60/crypt/zone file = sled-storage/src/dataset.rs:323
1452023-12-15T01:08:58.587ZINFOSledAgent (StorageManager): Loading latest secret disk_id = DiskIdentity {\n vendor: "synthetic-vendor",\n serial: "synthetic-serial-24b4dc87-ab46-49fb-a4b4-d361ae214c03",\n model: "synthetic-model",\n} file = sled-storage/src/dataset.rs:261
1462023-12-15T01:08:58.588ZINFOSledAgent (StorageManager): Loaded latest secret disk_id = DiskIdentity {\n vendor: "synthetic-vendor",\n serial: "synthetic-serial-24b4dc87-ab46-49fb-a4b4-d361ae214c03",\n model: "synthetic-model",\n} epoch = 0 file = sled-storage/src/dataset.rs:263
1472023-12-15T01:08:58.588ZINFOSledAgent (StorageManager): Retrieving key disk_id = DiskIdentity {\n vendor: "synthetic-vendor",\n serial: "synthetic-serial-24b4dc87-ab46-49fb-a4b4-d361ae214c03",\n model: "synthetic-model",\n} epoch = 0 file = sled-storage/src/dataset.rs:267
1482023-12-15T01:08:58.588ZINFOSledAgent (StorageManager): Got key disk_id = DiskIdentity {\n vendor: "synthetic-vendor",\n serial: "synthetic-serial-24b4dc87-ab46-49fb-a4b4-d361ae214c03",\n model: "synthetic-model",\n} epoch = 0 file = sled-storage/src/dataset.rs:269
1492023-12-15T01:08:58.588ZINFOSledAgent (StorageManager): Created keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-24b4dc87-ab46-49fb-a4b4-d361ae214c03-synthetic-model-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:37
1502023-12-15T01:08:58.588ZINFOSledAgent (StorageManager): Ensuring encrypted filesystem: crypt for epoch 0 file = sled-storage/src/dataset.rs:281
1512023-12-15T01:08:58.636ZINFOSledAgent (StorageManager): Zeroed and unlinked keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-24b4dc87-ab46-49fb-a4b4-d361ae214c03-synthetic-model-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:56
1522023-12-15T01:08:58.642ZINFOSledAgent (StorageManager): Automatically destroying dataset: oxp_24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone file = sled-storage/src/dataset.rs:323
1532023-12-15T01:08:58.789ZINFOSledAgent (StorageManager): Loading latest secret disk_id = DiskIdentity {\n vendor: "synthetic-vendor",\n serial: "synthetic-serial-e4b4dc87-ab46-49fb-a4b4-d361ae214c03",\n model: "synthetic-model",\n} file = sled-storage/src/dataset.rs:261
1542023-12-15T01:08:58.789ZINFOSledAgent (StorageManager): Loaded latest secret disk_id = DiskIdentity {\n vendor: "synthetic-vendor",\n serial: "synthetic-serial-e4b4dc87-ab46-49fb-a4b4-d361ae214c03",\n model: "synthetic-model",\n} epoch = 0 file = sled-storage/src/dataset.rs:263
1552023-12-15T01:08:58.789ZINFOSledAgent (StorageManager): Retrieving key disk_id = DiskIdentity {\n vendor: "synthetic-vendor",\n serial: "synthetic-serial-e4b4dc87-ab46-49fb-a4b4-d361ae214c03",\n model: "synthetic-model",\n} epoch = 0 file = sled-storage/src/dataset.rs:267
1562023-12-15T01:08:58.789ZINFOSledAgent (StorageManager): Got key disk_id = DiskIdentity {\n vendor: "synthetic-vendor",\n serial: "synthetic-serial-e4b4dc87-ab46-49fb-a4b4-d361ae214c03",\n model: "synthetic-model",\n} epoch = 0 file = sled-storage/src/dataset.rs:269
1572023-12-15T01:08:58.789ZINFOSledAgent (StorageManager): Created keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-e4b4dc87-ab46-49fb-a4b4-d361ae214c03-synthetic-model-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:37
1582023-12-15T01:08:58.789ZINFOSledAgent (StorageManager): Ensuring encrypted filesystem: crypt for epoch 0 file = sled-storage/src/dataset.rs:281
1592023-12-15T01:08:58.829ZINFOSledAgent (StorageManager): Zeroed and unlinked keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-e4b4dc87-ab46-49fb-a4b4-d361ae214c03-synthetic-model-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:56
1602023-12-15T01:08:58.835ZINFOSledAgent (StorageManager): Automatically destroying dataset: oxp_e4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone file = sled-storage/src/dataset.rs:323
1612023-12-15T01:08:59.004ZINFOSledAgent (StorageManager): Loading latest secret disk_id = DiskIdentity {\n vendor: "synthetic-vendor",\n serial: "synthetic-serial-31bd71cd-4736-4a12-a387-9b74b050396f",\n model: "synthetic-model",\n} file = sled-storage/src/dataset.rs:261
1622023-12-15T01:08:59.004ZINFOSledAgent (StorageManager): Loaded latest secret disk_id = DiskIdentity {\n vendor: "synthetic-vendor",\n serial: "synthetic-serial-31bd71cd-4736-4a12-a387-9b74b050396f",\n model: "synthetic-model",\n} epoch = 0 file = sled-storage/src/dataset.rs:263
1632023-12-15T01:08:59.004ZINFOSledAgent (StorageManager): Retrieving key disk_id = DiskIdentity {\n vendor: "synthetic-vendor",\n serial: "synthetic-serial-31bd71cd-4736-4a12-a387-9b74b050396f",\n model: "synthetic-model",\n} epoch = 0 file = sled-storage/src/dataset.rs:267
1642023-12-15T01:08:59.004ZINFOSledAgent (StorageManager): Got key disk_id = DiskIdentity {\n vendor: "synthetic-vendor",\n serial: "synthetic-serial-31bd71cd-4736-4a12-a387-9b74b050396f",\n model: "synthetic-model",\n} epoch = 0 file = sled-storage/src/dataset.rs:269
1652023-12-15T01:08:59.004ZINFOSledAgent (StorageManager): Created keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-31bd71cd-4736-4a12-a387-9b74b050396f-synthetic-model-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:37
1662023-12-15T01:08:59.004ZINFOSledAgent (StorageManager): Ensuring encrypted filesystem: crypt for epoch 0 file = sled-storage/src/dataset.rs:281
1672023-12-15T01:08:59.060ZINFOSledAgent (StorageManager): Zeroed and unlinked keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-31bd71cd-4736-4a12-a387-9b74b050396f-synthetic-model-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:56
1682023-12-15T01:08:59.067ZINFOSledAgent (StorageManager): Automatically destroying dataset: oxp_31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone file = sled-storage/src/dataset.rs:323
1692023-12-15T01:08:59.234ZINFOSledAgent (StorageManager): Loading latest secret disk_id = DiskIdentity {\n vendor: "synthetic-vendor",\n serial: "synthetic-serial-14b4dc87-ab46-49fb-a4b4-d361ae214c03",\n model: "synthetic-model",\n} file = sled-storage/src/dataset.rs:261
1702023-12-15T01:08:59.234ZINFOSledAgent (StorageManager): Loaded latest secret disk_id = DiskIdentity {\n vendor: "synthetic-vendor",\n serial: "synthetic-serial-14b4dc87-ab46-49fb-a4b4-d361ae214c03",\n model: "synthetic-model",\n} epoch = 0 file = sled-storage/src/dataset.rs:263
1712023-12-15T01:08:59.234ZINFOSledAgent (StorageManager): Retrieving key disk_id = DiskIdentity {\n vendor: "synthetic-vendor",\n serial: "synthetic-serial-14b4dc87-ab46-49fb-a4b4-d361ae214c03",\n model: "synthetic-model",\n} epoch = 0 file = sled-storage/src/dataset.rs:267
1722023-12-15T01:08:59.234ZINFOSledAgent (StorageManager): Got key disk_id = DiskIdentity {\n vendor: "synthetic-vendor",\n serial: "synthetic-serial-14b4dc87-ab46-49fb-a4b4-d361ae214c03",\n model: "synthetic-model",\n} epoch = 0 file = sled-storage/src/dataset.rs:269
1732023-12-15T01:08:59.234ZINFOSledAgent (StorageManager): Created keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-14b4dc87-ab46-49fb-a4b4-d361ae214c03-synthetic-model-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:37
1742023-12-15T01:08:59.234ZINFOSledAgent (StorageManager): Ensuring encrypted filesystem: crypt for epoch 0 file = sled-storage/src/dataset.rs:281
1752023-12-15T01:08:59.273ZINFOSledAgent (StorageManager): Zeroed and unlinked keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-14b4dc87-ab46-49fb-a4b4-d361ae214c03-synthetic-model-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:56
1762023-12-15T01:08:59.280ZINFOSledAgent (StorageManager): Automatically destroying dataset: oxp_14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone file = sled-storage/src/dataset.rs:323
1772023-12-15T01:08:59.431ZINFOSledAgent (StorageManager): Received GetLatestResources(Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) }) file = sled-storage/src/manager.rs:346
1782023-12-15T01:08:59.431ZINFOSledAgent (StorageManager): Received GetLatestResources(Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) }) file = sled-storage/src/manager.rs:346
1792023-12-15T01:08:59.431ZINFOSledAgent: Requested swap device of size 64 GiB file = sled-agent/src/sled_agent.rs:321 sled_id = 78829061-d92a-47ac-bcd6-186a0b9f0582
1802023-12-15T01:08:59.436ZINFOSledAgent (StorageMonitor): Received storage manager update file = sled-agent/src/storage_monitor.rs:111 resources = StorageResources { disks: {DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-14b4dc87-ab46-49fb-a4b4-d361ae214c03", model: "synthetic-model" }: (Synthetic(SyntheticDisk { identity: DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-14b4dc87-ab46-49fb-a4b4-d361ae214c03", model: "synthetic-model" }, zpool_name: ZpoolName { id: 14b4dc87-ab46-49fb-a4b4-d361ae214c03, kind: External } }), Pool { name: ZpoolName { id: 14b4dc87-ab46-49fb-a4b4-d361ae214c03, kind: External }, info: ZpoolInfo { name: "oxp_14b4dc87-ab46-49fb-a4b4-d361ae214c03", size: 20937965568, allocated: 772608, free: 20937192960, health: Online }, parent: DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-14b4dc87-ab46-49fb-a4b4-d361ae214c03", model: "synthetic-model" } }), DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-24b4dc87-ab46-49fb-a4b4-d361ae214c03", model: "synthetic-model" }: (Synthetic(SyntheticDisk { identity: DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-24b4dc87-ab46-49fb-a4b4-d361ae214c03", model: "synthetic-model" }, zpool_name: ZpoolName { id: 24b4dc87-ab46-49fb-a4b4-d361ae214c03, kind: External } }), Pool { name: ZpoolName { id: 24b4dc87-ab46-49fb-a4b4-d361ae214c03, kind: External }, info: ZpoolInfo { name: "oxp_24b4dc87-ab46-49fb-a4b4-d361ae214c03", size: 20937965568, allocated: 772608, free: 20937192960, health: Online }, parent: DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-24b4dc87-ab46-49fb-a4b4-d361ae214c03", model: "synthetic-model" } }), DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-31bd71cd-4736-4a12-a387-9b74b050396f", model: "synthetic-model" }: (Synthetic(SyntheticDisk { identity: DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-31bd71cd-4736-4a12-a387-9b74b050396f", model: "synthetic-model" }, zpool_name: ZpoolName { id: 31bd71cd-4736-4a12-a387-9b74b050396f, kind: External } }), Pool { name: ZpoolName { id: 31bd71cd-4736-4a12-a387-9b74b050396f, kind: External }, info: ZpoolInfo { name: "oxp_31bd71cd-4736-4a12-a387-9b74b050396f", size: 20937965568, allocated: 820224, free: 20937145344, health: Online }, parent: DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-31bd71cd-4736-4a12-a387-9b74b050396f", model: "synthetic-model" } }), DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-616b26df-e62a-4c68-b506-f4a923d8aaf7", model: "synthetic-model" }: (Synthetic(SyntheticDisk { identity: DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-616b26df-e62a-4c68-b506-f4a923d8aaf7", model: "synthetic-model" }, zpool_name: ZpoolName { id: 616b26df-e62a-4c68-b506-f4a923d8aaf7, kind: External } }), Pool { name: ZpoolName { id: 616b26df-e62a-4c68-b506-f4a923d8aaf7, kind: External }, info: ZpoolInfo { name: "oxp_616b26df-e62a-4c68-b506-f4a923d8aaf7", size: 20937965568, allocated: 1732608, free: 20936232960, health: Online }, parent: DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-616b26df-e62a-4c68-b506-f4a923d8aaf7", model: "synthetic-model" } }), DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-a462a7f7-b628-40fe-80ff-4e4189e2d62b", model: "synthetic-model" }: (Synthetic(SyntheticDisk { identity: DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-a462a7f7-b628-40fe-80ff-4e4189e2d62b", model: "synthetic-model" }, zpool_name: ZpoolName { id: a462a7f7-b628-40fe-80ff-4e4189e2d62b, kind: Internal } }), Pool { name: ZpoolName { id: a462a7f7-b628-40fe-80ff-4e4189e2d62b, kind: Internal }, info: ZpoolInfo { name: "oxi_a462a7f7-b628-40fe-80ff-4e4189e2d62b", size: 20937965568, allocated: 542208, free: 20937423360, health: Online }, parent: DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-a462a7f7-b628-40fe-80ff-4e4189e2d62b", model: "synthetic-model" } }), DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-b462a7f7-b628-40fe-80ff-4e4189e2d62b", model: "synthetic-model" }: (Synthetic(SyntheticDisk { identity: DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-b462a7f7-b628-40fe-80ff-4e4189e2d62b", model: "synthetic-model" }, zpool_name: ZpoolName { id: b462a7f7-b628-40fe-80ff-4e4189e2d62b, kind: Internal } }), Pool { name: ZpoolName { id: b462a7f7-b628-40fe-80ff-4e4189e2d62b, kind: Internal }, info: ZpoolInfo { name: "oxi_b462a7f7-b628-40fe-80ff-4e4189e2d62b", size: 20937965568, allocated: 493056, free: 20937472512, health: Online }, parent: DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-b462a7f7-b628-40fe-80ff-4e4189e2d62b", model: "synthetic-model" } }), DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-cd70d7f6-2354-4bf2-8012-55bf9eaf7930", model: "synthetic-model" }: (Synthetic(SyntheticDisk { identity: DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-cd70d7f6-2354-4bf2-8012-55bf9eaf7930", model: "synthetic-model" }, zpool_name: ZpoolName { id: cd70d7f6-2354-4bf2-8012-55bf9eaf7930, kind: External } }), Pool { name: ZpoolName { id: cd70d7f6-2354-4bf2-8012-55bf9eaf7930, kind: External }, info: ZpoolInfo { name: "oxp_cd70d7f6-2354-4bf2-8012-55bf9eaf7930", size: 20937965568, allocated: 778752, free: 20937186816, health: Online }, parent: DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-cd70d7f6-2354-4bf2-8012-55bf9eaf7930", model: "synthetic-model" } }), DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-ceb4461c-cf56-4719-ad3c-14430bfdfb60", model: "synthetic-model" }: (Synthetic(SyntheticDisk { identity: DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-ceb4461c-cf56-4719-ad3c-14430bfdfb60", model: "synthetic-model" }, zpool_name: ZpoolName { id: ceb4461c-cf56-4719-ad3c-14430bfdfb60, kind: External } }), Pool { name: ZpoolName { id: ceb4461c-cf56-4719-ad3c-14430bfdfb60, kind: External }, info: ZpoolInfo { name: "oxp_ceb4461c-cf56-4719-ad3c-14430bfdfb60", size: 20937965568, allocated: 769536, free: 20937196032, health: Online }, parent: DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-ceb4461c-cf56-4719-ad3c-14430bfdfb60", model: "synthetic-model" } }), DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-d462a7f7-b628-40fe-80ff-4e4189e2d62b", model: "synthetic-model" }: (Synthetic(SyntheticDisk { identity: DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-d462a7f7-b628-40fe-80ff-4e4189e2d62b", model: "synthetic-model" }, zpool_name: ZpoolName { id: d462a7f7-b628-40fe-80ff-4e4189e2d62b, kind: External } }), Pool { name: ZpoolName { id: d462a7f7-b628-40fe-80ff-4e4189e2d62b, kind: External }, info: ZpoolInfo { name: "oxp_d462a7f7-b628-40fe-80ff-4e4189e2d62b", size: 20937965568, allocated: 821760, free: 20937143808, health: Online }, parent: DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-d462a7f7-b628-40fe-80ff-4e4189e2d62b", model: "synthetic-model" } }), DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-e4b4dc87-ab46-49fb-a4b4-d361ae214c03", model: "synthetic-model" }: (Synthetic(SyntheticDisk { identity: DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-e4b4dc87-ab46-49fb-a4b4-d361ae214c03", model: "synthetic-model" }, zpool_name: ZpoolName { id: e4b4dc87-ab46-49fb-a4b4-d361ae214c03, kind: External } }), Pool { name: ZpoolName { id: e4b4dc87-ab46-49fb-a4b4-d361ae214c03, kind: External }, info: ZpoolInfo { name: "oxp_e4b4dc87-ab46-49fb-a4b4-d361ae214c03", size: 20937965568, allocated: 772608, free: 20937192960, health: Online }, parent: DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-e4b4dc87-ab46-49fb-a4b4-d361ae214c03", model: "synthetic-model" } }), DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-f4b4dc87-ab46-49fb-a4b4-d361ae214c03", model: "synthetic-model" }: (Synthetic(SyntheticDisk { identity: DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-f4b4dc87-ab46-49fb-a4b4-d361ae214c03", model: "synthetic-model" }, zpool_name: ZpoolName { id: f4b4dc87-ab46-49fb-a4b4-d361ae214c03, kind: External } }), Pool { name: ZpoolName { id: f4b4dc87-ab46-49fb-a4b4-d361ae214c03, kind: External }, info: ZpoolInfo { name: "oxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03", size: 20937965568, allocated: 772608, free: 20937192960, health: Online }, parent: DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-f4b4dc87-ab46-49fb-a4b4-d361ae214c03", model: "synthetic-model" } })} }
1812023-12-15T01:08:59.453ZINFOSledAgent: attempting to create encrypted zvol: name="oxi_a462a7f7-b628-40fe-80ff-4e4189e2d62b/swap", size_gb=64 file = sled-agent/src/swap_device.rs:153
1822023-12-15T01:08:59.456ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_switch file = illumos-utils/src/zone.rs:314
1832023-12-15T01:08:59.475ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_switch file = illumos-utils/src/zone.rs:353
1842023-12-15T01:08:59.500ZINFOSledAgent: successfully created encrypted zvol: name="oxi_a462a7f7-b628-40fe-80ff-4e4189e2d62b/swap", size_gb=64 file = sled-agent/src/swap_device.rs:249
1852023-12-15T01:08:59.500ZINFOSledAgent: adding swap device: swapname="/dev/zvol/dsk/oxi_a462a7f7-b628-40fe-80ff-4e4189e2d62b/swap" file = sled-agent/src/swap_device.rs:96
1862023-12-15T01:08:59.505ZINFOSledAgent: Mounting backing filesystems file = sled-agent/src/sled_agent.rs:336 sled_id = 78829061-d92a-47ac-bcd6-186a0b9f0582
1872023-12-15T01:08:59.505ZINFOSledAgent (BackingFs): Processing fmd file = sled-agent/src/backing_fs.rs:126
1882023-12-15T01:08:59.505ZINFOSledAgent (BackingFs): Ensuring dataset oxi_a462a7f7-b628-40fe-80ff-4e4189e2d62b/backing/fmd file = sled-agent/src/backing_fs.rs:136
1892023-12-15T01:08:59.587ZINFOSledAgent (BackingFs): Stopping service svc:/system/fmd:default file = sled-agent/src/backing_fs.rs:170
1902023-12-15T01:08:59.806ZINFOSledAgent (BackingFs): Mounting oxi_a462a7f7-b628-40fe-80ff-4e4189e2d62b/backing/fmd on /var/fm/fmd file = sled-agent/src/backing_fs.rs:178
1912023-12-15T01:08:59.819ZINFOSledAgent (BackingFs): Ensuring directory /var/fm/fmd/rsrc file = sled-agent/src/backing_fs.rs:186
1922023-12-15T01:08:59.819ZINFOSledAgent (BackingFs): Ensuring directory /var/fm/fmd/ckpt file = sled-agent/src/backing_fs.rs:186
1932023-12-15T01:08:59.819ZINFOSledAgent (BackingFs): Ensuring directory /var/fm/fmd/xprt file = sled-agent/src/backing_fs.rs:186
1942023-12-15T01:08:59.819ZINFOSledAgent (BackingFs): Starting service svc:/system/fmd:default file = sled-agent/src/backing_fs.rs:192
1952023-12-15T01:09:00.129ZINFOSledAgent: Ensuring contract reaper thread file = illumos-utils/src/running_zone.rs:162
1962023-12-15T01:09:00.657ZINFOSledAgent: using '[AddrObject { interface: "net0", name: "ll" }, AddrObject { interface: "net1", name: "ll" }]' as data links for xde driver file = illumos-utils/src/opte/illumos.rs:80 sled_id = 78829061-d92a-47ac-bcd6-186a0b9f0582
1972023-12-15T01:09:00.657ZINFOSledAgent (StorageMonitor): Underlay Available file = sled-agent/src/storage_monitor.rs:122 sled_id = 78829061-d92a-47ac-bcd6-186a0b9f0582
1982023-12-15T01:09:00.657ZINFOSledAgent (InstanceManager): 50% of 102952026112 physical ram = 51476013056 bytes) file = sled-agent/src/instance_manager.rs:177
1992023-12-15T01:09:00.658ZINFOSledAgent (InstanceManager): Setting reservoir size to 49090 MiB bytes file = sled-agent/src/instance_manager.rs:185
2002023-12-15T01:09:00.669ZINFOSledAgent (StorageMonitor): failed to notify nexus about Put(PhysicalDiskPutRequest { model: "synthetic-model", serial: "synthetic-serial-14b4dc87-ab46-49fb-a4b4-d361ae214c03", sled_id: 78829061-d92a-47ac-bcd6-186a0b9f0582, variant: U2, vendor: "synthetic-vendor" }) err = "Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/physical-disk): error trying to connect: dns error: proto error: io error: No route to host (os error 148)" file = sled-agent/src/storage_monitor.rs:219
2012023-12-15T01:09:00.669ZINFOSledAgent (StorageMonitor): failed to notify nexus about Put(PhysicalDiskPutRequest { model: "synthetic-model", serial: "synthetic-serial-a462a7f7-b628-40fe-80ff-4e4189e2d62b", sled_id: 78829061-d92a-47ac-bcd6-186a0b9f0582, variant: M2, vendor: "synthetic-vendor" }) err = "Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/physical-disk): error trying to connect: dns error: proto error: io error: No route to host (os error 148)" file = sled-agent/src/storage_monitor.rs:219
2022023-12-15T01:09:00.670ZINFOSledAgent (StorageMonitor): failed to notify nexus about Put(PhysicalDiskPutRequest { model: "synthetic-model", serial: "synthetic-serial-cd70d7f6-2354-4bf2-8012-55bf9eaf7930", sled_id: 78829061-d92a-47ac-bcd6-186a0b9f0582, variant: U2, vendor: "synthetic-vendor" }) err = "Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/physical-disk): error trying to connect: dns error: proto error: io error: No route to host (os error 148)" file = sled-agent/src/storage_monitor.rs:219
2032023-12-15T01:09:00.670ZINFOSledAgent (StorageMonitor): failed to notify nexus about a new pool oxp_24b4dc87-ab46-49fb-a4b4-d361ae214c03 on disk DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-24b4dc87-ab46-49fb-a4b4-d361ae214c03", model: "synthetic-model" } err = "Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/sled-agents/78829061-d92a-47ac-bcd6-186a0b9f0582/zpools/24b4dc87-ab46-49fb-a4b4-d361ae214c03): error trying to connect: dns error: proto error: io error: No route to host (os error 148)" file = sled-agent/src/storage_monitor.rs:270
2042023-12-15T01:09:00.670ZINFOSledAgent (StorageMonitor): failed to notify nexus about a new pool oxp_ceb4461c-cf56-4719-ad3c-14430bfdfb60 on disk DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-ceb4461c-cf56-4719-ad3c-14430bfdfb60", model: "synthetic-model" } err = "Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/sled-agents/78829061-d92a-47ac-bcd6-186a0b9f0582/zpools/ceb4461c-cf56-4719-ad3c-14430bfdfb60): error trying to connect: dns error: proto error: io error: No route to host (os error 148)" file = sled-agent/src/storage_monitor.rs:270
2052023-12-15T01:09:00.670ZINFOSledAgent (StorageMonitor): failed to notify nexus about Put(PhysicalDiskPutRequest { model: "synthetic-model", serial: "synthetic-serial-24b4dc87-ab46-49fb-a4b4-d361ae214c03", sled_id: 78829061-d92a-47ac-bcd6-186a0b9f0582, variant: U2, vendor: "synthetic-vendor" }) err = "Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/physical-disk): error trying to connect: dns error: proto error: io error: No route to host (os error 148)" file = sled-agent/src/storage_monitor.rs:219
2062023-12-15T01:09:00.671ZINFOSledAgent (StorageMonitor): failed to notify nexus about Put(PhysicalDiskPutRequest { model: "synthetic-model", serial: "synthetic-serial-31bd71cd-4736-4a12-a387-9b74b050396f", sled_id: 78829061-d92a-47ac-bcd6-186a0b9f0582, variant: U2, vendor: "synthetic-vendor" }) err = "Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/physical-disk): error trying to connect: dns error: proto error: io error: No route to host (os error 148)" file = sled-agent/src/storage_monitor.rs:219
2072023-12-15T01:09:00.671ZINFOSledAgent (StorageMonitor): failed to notify nexus about Put(PhysicalDiskPutRequest { model: "synthetic-model", serial: "synthetic-serial-616b26df-e62a-4c68-b506-f4a923d8aaf7", sled_id: 78829061-d92a-47ac-bcd6-186a0b9f0582, variant: U2, vendor: "synthetic-vendor" }) err = "Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/physical-disk): error trying to connect: dns error: proto error: io error: No route to host (os error 148)" file = sled-agent/src/storage_monitor.rs:219
2082023-12-15T01:09:00.671ZINFOSledAgent (StorageMonitor): failed to notify nexus about Put(PhysicalDiskPutRequest { model: "synthetic-model", serial: "synthetic-serial-b462a7f7-b628-40fe-80ff-4e4189e2d62b", sled_id: 78829061-d92a-47ac-bcd6-186a0b9f0582, variant: M2, vendor: "synthetic-vendor" }) err = "Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/physical-disk): error trying to connect: dns error: proto error: io error: No route to host (os error 148)" file = sled-agent/src/storage_monitor.rs:219
2092023-12-15T01:09:00.671ZINFOSledAgent (StorageMonitor): failed to notify nexus about Put(PhysicalDiskPutRequest { model: "synthetic-model", serial: "synthetic-serial-ceb4461c-cf56-4719-ad3c-14430bfdfb60", sled_id: 78829061-d92a-47ac-bcd6-186a0b9f0582, variant: U2, vendor: "synthetic-vendor" }) err = "Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/physical-disk): error trying to connect: dns error: proto error: io error: No route to host (os error 148)" file = sled-agent/src/storage_monitor.rs:219
2102023-12-15T01:09:00.671ZINFOSledAgent (StorageMonitor): failed to notify nexus about Put(PhysicalDiskPutRequest { model: "synthetic-model", serial: "synthetic-serial-d462a7f7-b628-40fe-80ff-4e4189e2d62b", sled_id: 78829061-d92a-47ac-bcd6-186a0b9f0582, variant: U2, vendor: "synthetic-vendor" }) err = "Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/physical-disk): error trying to connect: dns error: proto error: io error: No route to host (os error 148)" file = sled-agent/src/storage_monitor.rs:219
2112023-12-15T01:09:00.671ZINFOSledAgent (StorageMonitor): failed to notify nexus about Put(PhysicalDiskPutRequest { model: "synthetic-model", serial: "synthetic-serial-e4b4dc87-ab46-49fb-a4b4-d361ae214c03", sled_id: 78829061-d92a-47ac-bcd6-186a0b9f0582, variant: U2, vendor: "synthetic-vendor" }) err = "Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/physical-disk): error trying to connect: dns error: proto error: io error: No route to host (os error 148)" file = sled-agent/src/storage_monitor.rs:219
2122023-12-15T01:09:00.671ZINFOSledAgent (StorageMonitor): failed to notify nexus about Put(PhysicalDiskPutRequest { model: "synthetic-model", serial: "synthetic-serial-f4b4dc87-ab46-49fb-a4b4-d361ae214c03", sled_id: 78829061-d92a-47ac-bcd6-186a0b9f0582, variant: U2, vendor: "synthetic-vendor" }) err = "Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/physical-disk): error trying to connect: dns error: proto error: io error: No route to host (os error 148)" file = sled-agent/src/storage_monitor.rs:219
2132023-12-15T01:09:00.671ZINFOSledAgent (StorageMonitor): failed to notify nexus about a new pool oxp_14b4dc87-ab46-49fb-a4b4-d361ae214c03 on disk DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-14b4dc87-ab46-49fb-a4b4-d361ae214c03", model: "synthetic-model" } err = "Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/sled-agents/78829061-d92a-47ac-bcd6-186a0b9f0582/zpools/14b4dc87-ab46-49fb-a4b4-d361ae214c03): error trying to connect: dns error: proto error: io error: No route to host (os error 148)" file = sled-agent/src/storage_monitor.rs:270
2142023-12-15T01:09:00.671ZINFOSledAgent (StorageMonitor): failed to notify nexus about a new pool oxp_31bd71cd-4736-4a12-a387-9b74b050396f on disk DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-31bd71cd-4736-4a12-a387-9b74b050396f", model: "synthetic-model" } err = "Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/sled-agents/78829061-d92a-47ac-bcd6-186a0b9f0582/zpools/31bd71cd-4736-4a12-a387-9b74b050396f): error trying to connect: dns error: proto error: io error: No route to host (os error 148)" file = sled-agent/src/storage_monitor.rs:270
2152023-12-15T01:09:00.672ZINFOSledAgent (StorageMonitor): failed to notify nexus about a new pool oxp_616b26df-e62a-4c68-b506-f4a923d8aaf7 on disk DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-616b26df-e62a-4c68-b506-f4a923d8aaf7", model: "synthetic-model" } err = "Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/sled-agents/78829061-d92a-47ac-bcd6-186a0b9f0582/zpools/616b26df-e62a-4c68-b506-f4a923d8aaf7): error trying to connect: dns error: proto error: io error: No route to host (os error 148)" file = sled-agent/src/storage_monitor.rs:270
2162023-12-15T01:09:00.672ZINFOSledAgent (StorageMonitor): failed to notify nexus about a new pool oxi_a462a7f7-b628-40fe-80ff-4e4189e2d62b on disk DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-a462a7f7-b628-40fe-80ff-4e4189e2d62b", model: "synthetic-model" } err = "Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/sled-agents/78829061-d92a-47ac-bcd6-186a0b9f0582/zpools/a462a7f7-b628-40fe-80ff-4e4189e2d62b): error trying to connect: dns error: proto error: io error: No route to host (os error 148)" file = sled-agent/src/storage_monitor.rs:270
2172023-12-15T01:09:00.672ZINFOSledAgent (StorageMonitor): failed to notify nexus about a new pool oxi_b462a7f7-b628-40fe-80ff-4e4189e2d62b on disk DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-b462a7f7-b628-40fe-80ff-4e4189e2d62b", model: "synthetic-model" } err = "Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/sled-agents/78829061-d92a-47ac-bcd6-186a0b9f0582/zpools/b462a7f7-b628-40fe-80ff-4e4189e2d62b): error trying to connect: dns error: proto error: io error: No route to host (os error 148)" file = sled-agent/src/storage_monitor.rs:270
2182023-12-15T01:09:00.672ZINFOSledAgent (StorageMonitor): failed to notify nexus about a new pool oxp_cd70d7f6-2354-4bf2-8012-55bf9eaf7930 on disk DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-cd70d7f6-2354-4bf2-8012-55bf9eaf7930", model: "synthetic-model" } err = "Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/sled-agents/78829061-d92a-47ac-bcd6-186a0b9f0582/zpools/cd70d7f6-2354-4bf2-8012-55bf9eaf7930): error trying to connect: dns error: proto error: io error: No route to host (os error 148)" file = sled-agent/src/storage_monitor.rs:270
2192023-12-15T01:09:00.672ZINFOSledAgent (StorageMonitor): failed to notify nexus about a new pool oxp_d462a7f7-b628-40fe-80ff-4e4189e2d62b on disk DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-d462a7f7-b628-40fe-80ff-4e4189e2d62b", model: "synthetic-model" } err = "Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/sled-agents/78829061-d92a-47ac-bcd6-186a0b9f0582/zpools/d462a7f7-b628-40fe-80ff-4e4189e2d62b): error trying to connect: dns error: proto error: io error: No route to host (os error 148)" file = sled-agent/src/storage_monitor.rs:270
2202023-12-15T01:09:00.672ZINFOSledAgent (StorageMonitor): failed to notify nexus about a new pool oxp_e4b4dc87-ab46-49fb-a4b4-d361ae214c03 on disk DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-e4b4dc87-ab46-49fb-a4b4-d361ae214c03", model: "synthetic-model" } err = "Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/sled-agents/78829061-d92a-47ac-bcd6-186a0b9f0582/zpools/e4b4dc87-ab46-49fb-a4b4-d361ae214c03): error trying to connect: dns error: proto error: io error: No route to host (os error 148)" file = sled-agent/src/storage_monitor.rs:270
2212023-12-15T01:09:00.672ZINFOSledAgent (StorageMonitor): failed to notify nexus about a new pool oxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03 on disk DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-f4b4dc87-ab46-49fb-a4b4-d361ae214c03", model: "synthetic-model" } err = "Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/sled-agents/78829061-d92a-47ac-bcd6-186a0b9f0582/zpools/f4b4dc87-ab46-49fb-a4b4-d361ae214c03): error trying to connect: dns error: proto error: io error: No route to host (os error 148)" file = sled-agent/src/storage_monitor.rs:270
2222023-12-15T01:09:02.683ZINFOSledAgent (ServiceManager): Zone booting file = illumos-utils/src/running_zone.rs:512 zone = oxz_switch
2232023-12-15T01:09:02.870ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {}
2242023-12-15T01:09:03.071ZWARNSledAgent (ServiceManager): wait for service Some("oxz_switch") failed: Property not found. retry in 45.356269ms file = illumos-utils/src/svc.rs:36 zone = oxz_switch
2252023-12-15T01:09:03.291ZWARNSledAgent (ServiceManager): wait for service Some("oxz_switch") failed: Property not found. retry in 91.504688ms file = illumos-utils/src/svc.rs:36 zone = oxz_switch
2262023-12-15T01:09:03.473ZWARNSledAgent (ServiceManager): wait for service Some("oxz_switch") failed: Property not found. retry in 119.282161ms file = illumos-utils/src/svc.rs:36 zone = oxz_switch
2272023-12-15T01:09:03.652ZWARNSledAgent (ServiceManager): wait for service Some("oxz_switch") failed: Property not found. retry in 557.55464ms file = illumos-utils/src/svc.rs:36 zone = oxz_switch
2282023-12-15T01:09:04.227ZWARNSledAgent (ServiceManager): wait for service Some("oxz_switch") failed: Property not found. retry in 716.390525ms file = illumos-utils/src/svc.rs:36 zone = oxz_switch
2292023-12-15T01:09:04.994ZWARNSledAgent (ServiceManager): wait for service Some("oxz_switch") failed: Property not found. retry in 870.177534ms file = illumos-utils/src/svc.rs:36 zone = oxz_switch
2302023-12-15T01:09:05.877ZWARNSledAgent (ServiceManager): wait for service Some("oxz_switch") failed: Property not found. retry in 711.38515ms file = illumos-utils/src/svc.rs:36 zone = oxz_switch
2312023-12-15T01:09:06.665ZWARNSledAgent (ServiceManager): wait for service Some("oxz_switch") failed: Property not found. retry in 1.215107784s file = illumos-utils/src/svc.rs:36 zone = oxz_switch
2322023-12-15T01:09:07.872ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {}
2332023-12-15T01:09:09.066ZINFOSledAgent (ServiceManager): Ensuring tfportrear0_0/ll exists in zone file = sled-agent/src/services.rs:1711
2342023-12-15T01:09:10.717ZINFOSledAgent (ServiceManager): sled agent started file = sled-agent/src/services.rs:931 underlay_address = fd00:1122:3344:101::1
2352023-12-15T01:09:10.717ZINFOSledAgent (DnsResolver): new DNS resolver addresses = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53, [fd00:1122:3344:4::1]:53, [fd00:1122:3344:5::1]:53] file = internal-dns/src/resolver.rs:60
2362023-12-15T01:09:10.774ZINFOSledAgent (dropshot (SledAgent)): listening file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:195 local_addr = [fd00:1122:3344:101::1]:12345
2372023-12-15T01:09:10.774ZINFOSledAgent: contacting server nexus, registering sled baseboard = Baseboard { part_number: "i86pc", revision: 0, serial_number: "37fc7d027c1a4e9eb4f7b462ba03b0a1" } file = sled-agent/src/sled_agent.rs:637 id = 78829061-d92a-47ac-bcd6-186a0b9f0582 sled_id = 78829061-d92a-47ac-bcd6-186a0b9f0582
2382023-12-15T01:09:10.774ZINFOSledAgent (SledAgentMain): Sled Agent loaded; recording configuration file = sled-agent/src/bootstrap/server.rs:424
2392023-12-15T01:09:10.774ZINFOSledAgent (StorageManager): Received GetLatestResources(Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) }) file = sled-storage/src/manager.rs:346
2402023-12-15T01:09:10.775ZINFOSledAgent (SledAgentMain): Writing ledger to /pool/int/a462a7f7-b628-40fe-80ff-4e4189e2d62b/config/.sled-agent-request.json.tmp file = common/src/ledger.rs:198
2412023-12-15T01:09:10.775ZWARNSledAgent: failed to register as a metric producer with Nexus error = "Metric registration error: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/metrics/producers): error trying to connect: dns error: proto error: io error: No route to host (os error 148)" file = sled-agent/src/sled_agent.rs:1076 retry_after = 357.604225ms sled_id = 78829061-d92a-47ac-bcd6-186a0b9f0582
2422023-12-15T01:09:10.775ZINFOSledAgent (SledAgentMain): Writing ledger to /pool/int/b462a7f7-b628-40fe-80ff-4e4189e2d62b/config/.sled-agent-request.json.tmp file = common/src/ledger.rs:198
2432023-12-15T01:09:10.775ZINFOSledAgent (HardwareMonitor): Sled Agent Started file = sled-agent/src/hardware_monitor.rs:137
2442023-12-15T01:09:10.775ZINFOSledAgent (HardwareMonitor): Checking current full hardware snapshot file = sled-agent/src/hardware_monitor.rs:241 underlay_network_info = Some((fd00:1122:3344:101::2, Some(RackNetworkConfigV1 { rack_subnet: Ipv6Network { addr: fd00:1122:3344:1::, prefix: 56 }, infra_ip_first: 10.151.2.174, infra_ip_last: 10.151.2.174, ports: [PortConfigV1 { routes: [RouteConfig { destination: V4(Ipv4Network { addr: 0.0.0.0, prefix: 0 }), nexthop: 10.151.2.100 }], addresses: [V4(Ipv4Network { addr: 10.151.2.174, prefix: 32 })], switch: Switch0, port: "qsfp0", uplink_port_speed: Speed40G, uplink_port_fec: None, bgp_peers: [], autoneg: false }], bgp: [] })))
2452023-12-15T01:09:10.775ZINFOSledAgent (ServiceManager): Ensuring scrimlet services (enabling services) file = sled-agent/src/services.rs:3037
2462023-12-15T01:09:10.775ZINFOSledAgent (SledAgentSprocketsServer): Connection closed file = sled-agent/src/bootstrap/sprockets_server.rs:73 remote_addr = [fdb0:18c0:4d0d:a02a::1]:52055
2472023-12-15T01:09:10.778ZINFOSledAgent (BootstrapAgentRssHandler): Peer agent initialized file = sled-agent/src/bootstrap/rss_handle.rs:108 peer = [fdb0:18c0:4d0d:a02a::1]:12346
2482023-12-15T01:09:10.778ZINFOSledAgent (BootstrapAgentRssHandler): Initialized sled agent file = sled-agent/src/bootstrap/rss_handle.rs:232 target_sled = [fdb0:18c0:4d0d:a02a::1]:12346
2492023-12-15T01:09:10.778ZINFOSledAgent (StorageManager): Received GetLatestResources(Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) }) file = sled-storage/src/manager.rs:346
2502023-12-15T01:09:10.778ZINFOSledAgent (RSS): No ledger in /pool/int/a462a7f7-b628-40fe-80ff-4e4189e2d62b/config/rss-service-plan-v2.json file = common/src/ledger.rs:187
2512023-12-15T01:09:10.778ZINFOSledAgent (RSS): No ledger in /pool/int/b462a7f7-b628-40fe-80ff-4e4189e2d62b/config/rss-service-plan-v2.json file = common/src/ledger.rs:187
2522023-12-15T01:09:10.778ZINFOSledAgent (StorageManager): Received GetLatestResources(Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) }) file = sled-storage/src/manager.rs:346
2532023-12-15T01:09:10.779ZINFOSledAgent: failed to notify nexus about sled agent error = Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/sled-agents/78829061-d92a-47ac-bcd6-186a0b9f0582): error trying to connect: dns error: proto error: io error: No route to host (os error 148) file = sled-agent/src/sled_agent.rs:674 sled_id = 78829061-d92a-47ac-bcd6-186a0b9f0582
2542023-12-15T01:09:10.811ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:769 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:35008
2552023-12-15T01:09:10.811ZINFOSledAgent (StorageManager): Received GetLatestResources(Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) }) file = sled-storage/src/manager.rs:346
2562023-12-15T01:09:10.811ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 103 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:35008 req_id = b7c34810-a548-4baf-a909-04bd531a32e7 response_code = 200 uri = /zpools
2572023-12-15T01:09:10.841ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:769 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:45174
2582023-12-15T01:09:10.841ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 42 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:45174 req_id = af57a251-fbf4-415c-9398-9d2a84f1738e response_code = 200 uri = /sled-role
2592023-12-15T01:09:10.844ZINFOSledAgent (StorageManager): Received GetLatestResources(Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) }) file = sled-storage/src/manager.rs:346
2602023-12-15T01:09:10.844ZINFOSledAgent (RSS): Writing ledger to /pool/int/a462a7f7-b628-40fe-80ff-4e4189e2d62b/config/.rss-service-plan-v2.json.tmp file = common/src/ledger.rs:198
2612023-12-15T01:09:10.844ZINFOSledAgent (RSS): Writing ledger to /pool/int/b462a7f7-b628-40fe-80ff-4e4189e2d62b/config/.rss-service-plan-v2.json.tmp file = common/src/ledger.rs:198
2622023-12-15T01:09:10.844ZINFOSledAgent (RSS): Service plan written to storage file = sled-agent/src/rack_setup/plan/service.rs:749
2632023-12-15T01:09:10.873ZINFOSledAgent (RSS): attempting to set up sled's Omicron zones: OmicronZonesConfig { generation: Generation(2), zones: [OmicronZoneConfig { id: c8d17562-2f06-40f3-9824-1f4cb22a7b12, underlay_address: fd00:1122:3344:1::1, zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 14b4dc87-ab46-49fb-a4b4-d361ae214c03, kind: External } }, http_address: [fd00:1122:3344:1::1]:5353, dns_address: [fd00:1122:3344:1::1]:53, gz_address: fd00:1122:3344:1::2, gz_address_index: 0 } }, OmicronZoneConfig { id: aa7234de-3d19-4031-963b-712389d0323e, underlay_address: fd00:1122:3344:2::1, zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 24b4dc87-ab46-49fb-a4b4-d361ae214c03, kind: External } }, http_address: [fd00:1122:3344:2::1]:5353, dns_address: [fd00:1122:3344:2::1]:53, gz_address: fd00:1122:3344:2::2, gz_address_index: 1 } }, OmicronZoneConfig { id: cc39462d-3d1b-41ac-8720-1bae501935df, underlay_address: fd00:1122:3344:3::1, zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 31bd71cd-4736-4a12-a387-9b74b050396f, kind: External } }, http_address: [fd00:1122:3344:3::1]:5353, dns_address: [fd00:1122:3344:3::1]:53, gz_address: fd00:1122:3344:3::2, gz_address_index: 2 } }] } file = sled-agent/src/rack_setup/service.rs:299 sled_address = [fd00:1122:3344:101::1]:12345
2642023-12-15T01:09:10.873ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:769 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:34044
2652023-12-15T01:09:10.874ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: c8d17562-2f06-40f3-9824-1f4cb22a7b12, dataset_name: DatasetName { pool_name: ZpoolName { id: 14b4dc87-ab46-49fb-a4b4-d361ae214c03, kind: External }, kind: InternalDns }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } }) file = sled-storage/src/manager.rs:346
2662023-12-15T01:09:10.874ZINFOSledAgent (StorageManager): add_dataset: NewFilesystemRequest { dataset_id: c8d17562-2f06-40f3-9824-1f4cb22a7b12, dataset_name: DatasetName { pool_name: ZpoolName { id: 14b4dc87-ab46-49fb-a4b4-d361ae214c03, kind: External }, kind: InternalDns }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } } file = sled-storage/src/manager.rs:564
2672023-12-15T01:09:10.913ZINFOSledAgent: contacting server nexus, registering sled baseboard = Baseboard { part_number: "i86pc", revision: 0, serial_number: "37fc7d027c1a4e9eb4f7b462ba03b0a1" } file = sled-agent/src/sled_agent.rs:637 id = 78829061-d92a-47ac-bcd6-186a0b9f0582 sled_id = 78829061-d92a-47ac-bcd6-186a0b9f0582
2682023-12-15T01:09:10.947ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: aa7234de-3d19-4031-963b-712389d0323e, dataset_name: DatasetName { pool_name: ZpoolName { id: 24b4dc87-ab46-49fb-a4b4-d361ae214c03, kind: External }, kind: InternalDns }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } }) file = sled-storage/src/manager.rs:346
2692023-12-15T01:09:10.947ZINFOSledAgent (StorageManager): add_dataset: NewFilesystemRequest { dataset_id: aa7234de-3d19-4031-963b-712389d0323e, dataset_name: DatasetName { pool_name: ZpoolName { id: 24b4dc87-ab46-49fb-a4b4-d361ae214c03, kind: External }, kind: InternalDns }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } } file = sled-storage/src/manager.rs:564
2702023-12-15T01:09:11.014ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: cc39462d-3d1b-41ac-8720-1bae501935df, dataset_name: DatasetName { pool_name: ZpoolName { id: 31bd71cd-4736-4a12-a387-9b74b050396f, kind: External }, kind: InternalDns }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } }) file = sled-storage/src/manager.rs:346
2712023-12-15T01:09:11.014ZINFOSledAgent (StorageManager): add_dataset: NewFilesystemRequest { dataset_id: cc39462d-3d1b-41ac-8720-1bae501935df, dataset_name: DatasetName { pool_name: ZpoolName { id: 31bd71cd-4736-4a12-a387-9b74b050396f, kind: External }, kind: InternalDns }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } } file = sled-storage/src/manager.rs:564
2722023-12-15T01:09:11.050ZINFOSledAgent: contacting server nexus, registering sled baseboard = Baseboard { part_number: "i86pc", revision: 0, serial_number: "37fc7d027c1a4e9eb4f7b462ba03b0a1" } file = sled-agent/src/sled_agent.rs:637 id = 78829061-d92a-47ac-bcd6-186a0b9f0582 sled_id = 78829061-d92a-47ac-bcd6-186a0b9f0582
2732023-12-15T01:09:11.086ZINFOSledAgent (StorageManager): Received GetLatestResources(Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) }) file = sled-storage/src/manager.rs:346
2742023-12-15T01:09:11.086ZINFOSledAgent (ServiceManager): No ledger in /pool/int/a462a7f7-b628-40fe-80ff-4e4189e2d62b/config/omicron-zones.json file = common/src/ledger.rs:187
2752023-12-15T01:09:11.086ZINFOSledAgent (ServiceManager): No ledger in /pool/int/b462a7f7-b628-40fe-80ff-4e4189e2d62b/config/omicron-zones.json file = common/src/ledger.rs:187
2762023-12-15T01:09:11.086ZINFOSledAgent (StorageManager): Received GetLatestResources(Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) }) file = sled-storage/src/manager.rs:346
2772023-12-15T01:09:11.091ZINFOSledAgent (StorageManager): Received GetLatestResources(Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) }) file = sled-storage/src/manager.rs:346
2782023-12-15T01:09:11.091ZINFOSledAgent (StorageManager): Received GetLatestResources(Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) }) file = sled-storage/src/manager.rs:346
2792023-12-15T01:09:11.091ZINFOSledAgent (StorageManager): Received GetLatestResources(Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) }) file = sled-storage/src/manager.rs:346
2802023-12-15T01:09:11.139ZWARNSledAgent: failed to register as a metric producer with Nexus error = "Metric registration error: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/metrics/producers): error trying to connect: dns error: proto error: io error: No route to host (os error 148)" file = sled-agent/src/sled_agent.rs:1076 retry_after = 330.259916ms sled_id = 78829061-d92a-47ac-bcd6-186a0b9f0582
2812023-12-15T01:09:11.146ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_internal_dns_cc39462d-3d1b-41ac-8720-1bae501935df file = illumos-utils/src/zone.rs:314
2822023-12-15T01:09:11.150ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_internal_dns_aa7234de-3d19-4031-963b-712389d0323e file = illumos-utils/src/zone.rs:314
2832023-12-15T01:09:11.154ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_internal_dns_c8d17562-2f06-40f3-9824-1f4cb22a7b12 file = illumos-utils/src/zone.rs:314
2842023-12-15T01:09:11.167ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_internal_dns_cc39462d-3d1b-41ac-8720-1bae501935df file = illumos-utils/src/zone.rs:353
2852023-12-15T01:09:11.172ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_internal_dns_aa7234de-3d19-4031-963b-712389d0323e file = illumos-utils/src/zone.rs:353
2862023-12-15T01:09:11.176ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_internal_dns_c8d17562-2f06-40f3-9824-1f4cb22a7b12 file = illumos-utils/src/zone.rs:353
2872023-12-15T01:09:11.247ZINFOSledAgent: contacting server nexus, registering sled baseboard = Baseboard { part_number: "i86pc", revision: 0, serial_number: "37fc7d027c1a4e9eb4f7b462ba03b0a1" } file = sled-agent/src/sled_agent.rs:637 id = 78829061-d92a-47ac-bcd6-186a0b9f0582 sled_id = 78829061-d92a-47ac-bcd6-186a0b9f0582
2882023-12-15T01:09:11.426ZINFOSledAgent: contacting server nexus, registering sled baseboard = Baseboard { part_number: "i86pc", revision: 0, serial_number: "37fc7d027c1a4e9eb4f7b462ba03b0a1" } file = sled-agent/src/sled_agent.rs:637 id = 78829061-d92a-47ac-bcd6-186a0b9f0582 sled_id = 78829061-d92a-47ac-bcd6-186a0b9f0582
2892023-12-15T01:09:11.470ZWARNSledAgent: failed to register as a metric producer with Nexus error = "Metric registration error: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/metrics/producers): error trying to connect: dns error: proto error: io error: No route to host (os error 148)" file = sled-agent/src/sled_agent.rs:1076 retry_after = 1.18583775s sled_id = 78829061-d92a-47ac-bcd6-186a0b9f0582
2902023-12-15T01:09:11.695ZINFOSledAgent: contacting server nexus, registering sled baseboard = Baseboard { part_number: "i86pc", revision: 0, serial_number: "37fc7d027c1a4e9eb4f7b462ba03b0a1" } file = sled-agent/src/sled_agent.rs:637 id = 78829061-d92a-47ac-bcd6-186a0b9f0582 sled_id = 78829061-d92a-47ac-bcd6-186a0b9f0582
2912023-12-15T01:09:11.854ZINFOSledAgent: contacting server nexus, registering sled baseboard = Baseboard { part_number: "i86pc", revision: 0, serial_number: "37fc7d027c1a4e9eb4f7b462ba03b0a1" } file = sled-agent/src/sled_agent.rs:637 id = 78829061-d92a-47ac-bcd6-186a0b9f0582 sled_id = 78829061-d92a-47ac-bcd6-186a0b9f0582
2922023-12-15T01:09:12.126ZINFOSledAgent: contacting server nexus, registering sled baseboard = Baseboard { part_number: "i86pc", revision: 0, serial_number: "37fc7d027c1a4e9eb4f7b462ba03b0a1" } file = sled-agent/src/sled_agent.rs:637 id = 78829061-d92a-47ac-bcd6-186a0b9f0582 sled_id = 78829061-d92a-47ac-bcd6-186a0b9f0582
2932023-12-15T01:09:12.404ZINFOSledAgent: contacting server nexus, registering sled baseboard = Baseboard { part_number: "i86pc", revision: 0, serial_number: "37fc7d027c1a4e9eb4f7b462ba03b0a1" } file = sled-agent/src/sled_agent.rs:637 id = 78829061-d92a-47ac-bcd6-186a0b9f0582 sled_id = 78829061-d92a-47ac-bcd6-186a0b9f0582
2942023-12-15T01:09:12.657ZWARNSledAgent: failed to register as a metric producer with Nexus error = "Metric registration error: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/metrics/producers): error trying to connect: dns error: proto error: io error: No route to host (os error 148)" file = sled-agent/src/sled_agent.rs:1076 retry_after = 2.239404421s sled_id = 78829061-d92a-47ac-bcd6-186a0b9f0582
2952023-12-15T01:09:12.867ZINFOSledAgent: contacting server nexus, registering sled baseboard = Baseboard { part_number: "i86pc", revision: 0, serial_number: "37fc7d027c1a4e9eb4f7b462ba03b0a1" } file = sled-agent/src/sled_agent.rs:637 id = 78829061-d92a-47ac-bcd6-186a0b9f0582 sled_id = 78829061-d92a-47ac-bcd6-186a0b9f0582
2962023-12-15T01:09:12.873ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {}
2972023-12-15T01:09:12.994ZINFOSledAgent (ServiceManager): Zone booting file = illumos-utils/src/running_zone.rs:512 zone = oxz_internal_dns_cc39462d-3d1b-41ac-8720-1bae501935df
2982023-12-15T01:09:13.035ZINFOSledAgent (ServiceManager): Zone booting file = illumos-utils/src/running_zone.rs:512 zone = oxz_internal_dns_aa7234de-3d19-4031-963b-712389d0323e
2992023-12-15T01:09:13.040ZINFOSledAgent (ServiceManager): Zone booting file = illumos-utils/src/running_zone.rs:512 zone = oxz_internal_dns_c8d17562-2f06-40f3-9824-1f4cb22a7b12
3002023-12-15T01:09:13.133ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_cc39462d-3d1b-41ac-8720-1bae501935df") failed: Property not found. retry in 48.744448ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_cc39462d-3d1b-41ac-8720-1bae501935df
3012023-12-15T01:09:13.165ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_aa7234de-3d19-4031-963b-712389d0323e") failed: Property not found. retry in 42.963844ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_aa7234de-3d19-4031-963b-712389d0323e
3022023-12-15T01:09:13.172ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_c8d17562-2f06-40f3-9824-1f4cb22a7b12") failed: Property not found. retry in 52.007369ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_c8d17562-2f06-40f3-9824-1f4cb22a7b12
3032023-12-15T01:09:13.190ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_cc39462d-3d1b-41ac-8720-1bae501935df") failed: Property not found. retry in 118.471922ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_cc39462d-3d1b-41ac-8720-1bae501935df
3042023-12-15T01:09:13.222ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_aa7234de-3d19-4031-963b-712389d0323e") failed: Property not found. retry in 114.922165ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_aa7234de-3d19-4031-963b-712389d0323e
3052023-12-15T01:09:13.355ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_c8d17562-2f06-40f3-9824-1f4cb22a7b12") failed: Property not found. retry in 82.010899ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_c8d17562-2f06-40f3-9824-1f4cb22a7b12
3062023-12-15T01:09:13.400ZINFOSledAgent: contacting server nexus, registering sled baseboard = Baseboard { part_number: "i86pc", revision: 0, serial_number: "37fc7d027c1a4e9eb4f7b462ba03b0a1" } file = sled-agent/src/sled_agent.rs:637 id = 78829061-d92a-47ac-bcd6-186a0b9f0582 sled_id = 78829061-d92a-47ac-bcd6-186a0b9f0582
3072023-12-15T01:09:13.437ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_cc39462d-3d1b-41ac-8720-1bae501935df") failed: Property not found. retry in 236.008082ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_cc39462d-3d1b-41ac-8720-1bae501935df
3082023-12-15T01:09:13.472ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_aa7234de-3d19-4031-963b-712389d0323e") failed: Property not found. retry in 178.63675ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_aa7234de-3d19-4031-963b-712389d0323e
3092023-12-15T01:09:13.586ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_c8d17562-2f06-40f3-9824-1f4cb22a7b12") failed: Property not found. retry in 162.48277ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_c8d17562-2f06-40f3-9824-1f4cb22a7b12
3102023-12-15T01:09:13.684ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_cc39462d-3d1b-41ac-8720-1bae501935df") failed: Property not found. retry in 430.794916ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_cc39462d-3d1b-41ac-8720-1bae501935df
3112023-12-15T01:09:13.714ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_aa7234de-3d19-4031-963b-712389d0323e") failed: Property not found. retry in 595.900287ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_aa7234de-3d19-4031-963b-712389d0323e
3122023-12-15T01:09:13.776ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_c8d17562-2f06-40f3-9824-1f4cb22a7b12") failed: Property not found. retry in 459.872695ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_c8d17562-2f06-40f3-9824-1f4cb22a7b12
3132023-12-15T01:09:14.122ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_cc39462d-3d1b-41ac-8720-1bae501935df") failed: Property not found. retry in 761.74153ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_cc39462d-3d1b-41ac-8720-1bae501935df
3142023-12-15T01:09:14.185ZINFOSledAgent: contacting server nexus, registering sled baseboard = Baseboard { part_number: "i86pc", revision: 0, serial_number: "37fc7d027c1a4e9eb4f7b462ba03b0a1" } file = sled-agent/src/sled_agent.rs:637 id = 78829061-d92a-47ac-bcd6-186a0b9f0582 sled_id = 78829061-d92a-47ac-bcd6-186a0b9f0582
3152023-12-15T01:09:14.203ZINFOSledAgent (ServiceManager): Ensuring bootstrap address fdb0:18c0:4d0d:a02a::2 exists in switch zone file = sled-agent/src/services.rs:1728
3162023-12-15T01:09:14.203ZINFOSledAgent (ServiceManager): Adding bootstrap address file = illumos-utils/src/running_zone.rs:625 zone = oxz_switch
3172023-12-15T01:09:14.246ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_c8d17562-2f06-40f3-9824-1f4cb22a7b12") failed: Property not found. retry in 739.98075ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_c8d17562-2f06-40f3-9824-1f4cb22a7b12
3182023-12-15T01:09:14.319ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_aa7234de-3d19-4031-963b-712389d0323e") failed: Property not found. retry in 1.064887051s file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_aa7234de-3d19-4031-963b-712389d0323e
3192023-12-15T01:09:14.894ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_cc39462d-3d1b-41ac-8720-1bae501935df") failed: Property not found. retry in 636.046648ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_cc39462d-3d1b-41ac-8720-1bae501935df
3202023-12-15T01:09:14.898ZWARNSledAgent: failed to register as a metric producer with Nexus error = "Metric registration error: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/metrics/producers): error trying to connect: dns error: proto error: io error: No route to host (os error 148)" file = sled-agent/src/sled_agent.rs:1076 retry_after = 4.963280608s sled_id = 78829061-d92a-47ac-bcd6-186a0b9f0582
3212023-12-15T01:09:14.994ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_c8d17562-2f06-40f3-9824-1f4cb22a7b12") failed: Property not found. retry in 1.433270249s file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_c8d17562-2f06-40f3-9824-1f4cb22a7b12
3222023-12-15T01:09:15.120ZINFOSledAgent: contacting server nexus, registering sled baseboard = Baseboard { part_number: "i86pc", revision: 0, serial_number: "37fc7d027c1a4e9eb4f7b462ba03b0a1" } file = sled-agent/src/sled_agent.rs:637 id = 78829061-d92a-47ac-bcd6-186a0b9f0582 sled_id = 78829061-d92a-47ac-bcd6-186a0b9f0582
3232023-12-15T01:09:15.394ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_aa7234de-3d19-4031-963b-712389d0323e") failed: Property not found. retry in 1.117517984s file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_aa7234de-3d19-4031-963b-712389d0323e
3242023-12-15T01:09:15.539ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_cc39462d-3d1b-41ac-8720-1bae501935df") failed: Property not found. retry in 651.97432ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_cc39462d-3d1b-41ac-8720-1bae501935df
3252023-12-15T01:09:15.631ZINFOSledAgent: contacting server nexus, registering sled baseboard = Baseboard { part_number: "i86pc", revision: 0, serial_number: "37fc7d027c1a4e9eb4f7b462ba03b0a1" } file = sled-agent/src/sled_agent.rs:637 id = 78829061-d92a-47ac-bcd6-186a0b9f0582 sled_id = 78829061-d92a-47ac-bcd6-186a0b9f0582
3262023-12-15T01:09:16.201ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_cc39462d-3d1b-41ac-8720-1bae501935df") failed: Property not found. retry in 552.288713ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_cc39462d-3d1b-41ac-8720-1bae501935df
3272023-12-15T01:09:16.437ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_c8d17562-2f06-40f3-9824-1f4cb22a7b12") failed: Property not found. retry in 950.23315ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_c8d17562-2f06-40f3-9824-1f4cb22a7b12
3282023-12-15T01:09:16.521ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_aa7234de-3d19-4031-963b-712389d0323e") failed: Property not found. retry in 1.404850638s file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_aa7234de-3d19-4031-963b-712389d0323e
3292023-12-15T01:09:16.762ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_cc39462d-3d1b-41ac-8720-1bae501935df") failed: Property not found. retry in 501.103339ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_cc39462d-3d1b-41ac-8720-1bae501935df
3302023-12-15T01:09:16.801ZINFOSledAgent: contacting server nexus, registering sled baseboard = Baseboard { part_number: "i86pc", revision: 0, serial_number: "37fc7d027c1a4e9eb4f7b462ba03b0a1" } file = sled-agent/src/sled_agent.rs:637 id = 78829061-d92a-47ac-bcd6-186a0b9f0582 sled_id = 78829061-d92a-47ac-bcd6-186a0b9f0582
3312023-12-15T01:09:17.271ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_cc39462d-3d1b-41ac-8720-1bae501935df") failed: Property not found. retry in 1.018249227s file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_cc39462d-3d1b-41ac-8720-1bae501935df
3322023-12-15T01:09:17.396ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_c8d17562-2f06-40f3-9824-1f4cb22a7b12") failed: Property not found. retry in 872.328525ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_c8d17562-2f06-40f3-9824-1f4cb22a7b12
3332023-12-15T01:09:17.873ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {}
3342023-12-15T01:09:17.935ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_aa7234de-3d19-4031-963b-712389d0323e") failed: Property not found. retry in 1.286277306s file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_aa7234de-3d19-4031-963b-712389d0323e
3352023-12-15T01:09:18.171ZINFOSledAgent: contacting server nexus, registering sled baseboard = Baseboard { part_number: "i86pc", revision: 0, serial_number: "37fc7d027c1a4e9eb4f7b462ba03b0a1" } file = sled-agent/src/sled_agent.rs:637 id = 78829061-d92a-47ac-bcd6-186a0b9f0582 sled_id = 78829061-d92a-47ac-bcd6-186a0b9f0582
3362023-12-15T01:09:18.278ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_c8d17562-2f06-40f3-9824-1f4cb22a7b12") failed: Property not found. retry in 1.201374888s file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_c8d17562-2f06-40f3-9824-1f4cb22a7b12
3372023-12-15T01:09:18.297ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_cc39462d-3d1b-41ac-8720-1bae501935df") failed: Property not found. retry in 1.143494035s file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_cc39462d-3d1b-41ac-8720-1bae501935df
3382023-12-15T01:09:18.742ZINFOSledAgent (ServiceManager): Forwarding bootstrap traffic via oxBootstrap0 to fe80::8:20ff:fe56:8ba1 file = sled-agent/src/services.rs:1735
3392023-12-15T01:09:18.748ZINFOSledAgent (ServiceManager): Zone not using gateway (even though underlay is up) file = sled-agent/src/services.rs:1796
3402023-12-15T01:09:18.875ZINFOSledAgent (ServiceManager): Setting up dendrite service file = sled-agent/src/services.rs:2275
3412023-12-15T01:09:19.133ZINFOSledAgent (ServiceManager): Setting up MGS service file = sled-agent/src/services.rs:2154
3422023-12-15T01:09:19.274ZINFOSledAgent (ServiceManager): Ensuring address fd00:1122:3344:2::1 exists file = sled-agent/src/services.rs:1764
3432023-12-15T01:09:19.274ZINFOSledAgent (ServiceManager): Adding address: Static(V6(Ipv6Network { addr: fd00:1122:3344:2::1, prefix: 64 })) file = illumos-utils/src/running_zone.rs:608 zone = oxz_internal_dns_aa7234de-3d19-4031-963b-712389d0323e
3442023-12-15T01:09:19.561ZINFOSledAgent (ServiceManager): Setting up wicketd service file = sled-agent/src/services.rs:2189
3452023-12-15T01:09:19.811ZINFOSledAgent (ServiceManager): Setting up mgd service file = sled-agent/src/services.rs:2456
3462023-12-15T01:09:19.835ZINFOSledAgent (ServiceManager): Ensuring address fd00:1122:3344:2::1 exists - OK file = sled-agent/src/services.rs:1772
3472023-12-15T01:09:19.835ZINFOSledAgent (ServiceManager): Zone not using gateway (even though underlay is up) file = sled-agent/src/services.rs:1796
3482023-12-15T01:09:19.864ZWARNSledAgent: failed to register as a metric producer with Nexus error = "Metric registration error: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/metrics/producers): error trying to connect: dns error: proto error: io error: No route to host (os error 148)" file = sled-agent/src/sled_agent.rs:1076 retry_after = 8.866776329s sled_id = 78829061-d92a-47ac-bcd6-186a0b9f0582
3492023-12-15T01:09:19.983ZINFOSledAgent (ServiceManager): Setting up mg-ddm service file = sled-agent/src/services.rs:2461
3502023-12-15T01:09:20.161ZINFOSledAgent (ServiceManager): Setting up tfport service file = sled-agent/src/services.rs:2395
3512023-12-15T01:09:20.201ZINFOSledAgent (ServiceManager): Setting up internal-dns service file = sled-agent/src/services.rs:1990
3522023-12-15T01:09:20.330ZINFOSledAgent (ServiceManager): Setting up Simulated SP service file = sled-agent/src/services.rs:2183
3532023-12-15T01:09:20.339ZINFOSledAgent (ServiceManager): Re-enabling running switch zone (new address) file = sled-agent/src/services.rs:3289 new = [fd00:1122:3344:101::2, ::1] old = [::1]
3542023-12-15T01:09:20.339ZINFOSledAgent (ServiceManager): Ensuring address fd00:1122:3344:101::2 exists file = sled-agent/src/services.rs:3305
3552023-12-15T01:09:20.339ZINFOSledAgent (ServiceManager): Adding address: Static(V6(Ipv6Network { addr: fd00:1122:3344:101::2, prefix: 64 })) file = illumos-utils/src/running_zone.rs:608 zone = oxz_switch
3562023-12-15T01:09:20.349ZINFOSledAgent: contacting server nexus, registering sled baseboard = Baseboard { part_number: "i86pc", revision: 0, serial_number: "37fc7d027c1a4e9eb4f7b462ba03b0a1" } file = sled-agent/src/sled_agent.rs:637 id = 78829061-d92a-47ac-bcd6-186a0b9f0582 sled_id = 78829061-d92a-47ac-bcd6-186a0b9f0582
3572023-12-15T01:09:20.763ZINFOSledAgent (BootstrapAgentStartup): Sending prefix to ddmd for advertisement DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:91 prefix = Ipv6Prefix { addr: fd00:1122:3344:2::, len: 64 }
3582023-12-15T01:09:20.797ZINFOSledAgent (ServiceManager): Ensuring address fd00:1122:3344:1::1 exists file = sled-agent/src/services.rs:1764
3592023-12-15T01:09:20.797ZINFOSledAgent (ServiceManager): Adding address: Static(V6(Ipv6Network { addr: fd00:1122:3344:1::1, prefix: 64 })) file = illumos-utils/src/running_zone.rs:608 zone = oxz_internal_dns_c8d17562-2f06-40f3-9824-1f4cb22a7b12
3602023-12-15T01:09:20.888ZINFOSledAgent (ServiceManager): Ensuring address fd00:1122:3344:101::2 exists - OK file = sled-agent/src/services.rs:3313
3612023-12-15T01:09:20.894ZINFOSledAgent (ServiceManager): configuring dendrite zone file = sled-agent/src/services.rs:3369
3622023-12-15T01:09:21.059ZINFOSledAgent (ServiceManager): configuring wicketd file = sled-agent/src/services.rs:3409 rack_subnet = fd00:1122:3344::
3632023-12-15T01:09:21.110ZINFOSledAgent (ServiceManager): Determining physical location of our switch zone at fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:359
3642023-12-15T01:09:21.143ZWARNSledAgent (ServiceManager): Failed to get switch ID from MGS (retrying in 71.285878ms) error = Error Response: status: 503 Service Unavailable; headers: {"content-type": "application/json", "x-request-id": "ea902d4a-107e-4238-b9d6-cedc75d1ef8d", "content-length": "152", "date": "Fri, 15 Dec 2023 01:09:20 GMT"}; value: Error { error_code: Some("DiscoveryNotYetComplete"), message: "discovery process not yet complete", request_id: "ea902d4a-107e-4238-b9d6-cedc75d1ef8d" } file = sled-agent/src/bootstrap/early_networking.rs:378
3652023-12-15T01:09:21.217ZWARNSledAgent (ServiceManager): Failed to get switch ID from MGS (retrying in 111.6049ms) error = Error Response: status: 503 Service Unavailable; headers: {"content-type": "application/json", "x-request-id": "0b3f6190-0d80-4bb5-8349-f556fb5d419a", "content-length": "152", "date": "Fri, 15 Dec 2023 01:09:21 GMT"}; value: Error { error_code: Some("DiscoveryNotYetComplete"), message: "discovery process not yet complete", request_id: "0b3f6190-0d80-4bb5-8349-f556fb5d419a" } file = sled-agent/src/bootstrap/early_networking.rs:378
3662023-12-15T01:09:21.330ZINFOSledAgent (ServiceManager): Initializing 1 Uplinks on Switch0 at fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:407
3672023-12-15T01:09:21.346ZINFOSledAgent (ServiceManager): Ensuring address fd00:1122:3344:1::1 exists - OK file = sled-agent/src/services.rs:1772
3682023-12-15T01:09:21.346ZINFOSledAgent (ServiceManager): Zone not using gateway (even though underlay is up) file = sled-agent/src/services.rs:1796
3692023-12-15T01:09:21.366ZINFOSledAgent (ServiceManager): Building Port Configuration file = sled-agent/src/bootstrap/early_networking.rs:558
3702023-12-15T01:09:21.366ZINFOSledAgent (ServiceManager): Checking dendrite uptime file = sled-agent/src/bootstrap/early_networking.rs:627
3712023-12-15T01:09:21.367ZINFOSledAgent (ServiceManager): Dendrite online file = sled-agent/src/bootstrap/early_networking.rs:630 uptime = 2
3722023-12-15T01:09:21.367ZINFOSledAgent (ServiceManager): Configuring boundary services loopback address on switch config = Ipv6Entry {\n addr: fd00:99::1,\n tag: "omicron",\n} file = sled-agent/src/bootstrap/early_networking.rs:429
3732023-12-15T01:09:21.367ZINFOSledAgent (ServiceManager): Configuring default uplink on switch config = PortSettings {\n links: {\n "0": LinkSettings {\n addrs: [\n 10.151.2.174,\n ],\n params: LinkCreate {\n autoneg: false,\n fec: None,\n kr: false,\n lane: Some(\n LinkId(\n 0,\n ),\n ),\n speed: Speed40G,\n },\n },\n },\n v4_routes: {\n "0.0.0.0/0": [\n RouteSettingsV4 {\n link_id: 0,\n nexthop: 10.151.2.100,\n },\n ],\n },\n v6_routes: {},\n} file = sled-agent/src/bootstrap/early_networking.rs:440
3742023-12-15T01:09:21.368ZINFOSledAgent (ServiceManager): advertising boundary services loopback address file = sled-agent/src/bootstrap/early_networking.rs:457
3752023-12-15T01:09:21.433ZINFOSledAgent (ServiceManager): Sending prefix to ddmd for advertisement DdmAdminClient = [fd00:1122:3344:101::2]:8000 file = clients/ddm-admin-client/src/lib.rs:91 prefix = Ipv6Prefix { addr: fd00:99::, len: 64 }
3762023-12-15T01:09:21.477ZINFOSledAgent (StorageManager): Received DisksChanged({}) file = sled-storage/src/manager.rs:346
3772023-12-15T01:09:21.603ZINFOSledAgent (ServiceManager): Setting up internal-dns service file = sled-agent/src/services.rs:1990
3782023-12-15T01:09:22.166ZINFOSledAgent (BootstrapAgentStartup): Sending prefix to ddmd for advertisement DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:91 prefix = Ipv6Prefix { addr: fd00:1122:3344:1::, len: 64 }
3792023-12-15T01:09:22.189ZINFOSledAgent (ServiceManager): Ensuring address fd00:1122:3344:3::1 exists file = sled-agent/src/services.rs:1764
3802023-12-15T01:09:22.189ZINFOSledAgent (ServiceManager): Adding address: Static(V6(Ipv6Network { addr: fd00:1122:3344:3::1, prefix: 64 })) file = illumos-utils/src/running_zone.rs:608 zone = oxz_internal_dns_cc39462d-3d1b-41ac-8720-1bae501935df
3812023-12-15T01:09:22.738ZINFOSledAgent (ServiceManager): Ensuring address fd00:1122:3344:3::1 exists - OK file = sled-agent/src/services.rs:1772
3822023-12-15T01:09:22.738ZINFOSledAgent (ServiceManager): Zone not using gateway (even though underlay is up) file = sled-agent/src/services.rs:1796
3832023-12-15T01:09:22.876ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::605a:15ff:fea0:c066": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
3842023-12-15T01:09:23.002ZINFOSledAgent (ServiceManager): Setting up internal-dns service file = sled-agent/src/services.rs:1990
3852023-12-15T01:09:23.555ZINFOSledAgent (ServiceManager): Writing ledger to /pool/int/a462a7f7-b628-40fe-80ff-4e4189e2d62b/config/.omicron-zones.json.tmp file = common/src/ledger.rs:198
3862023-12-15T01:09:23.555ZINFOSledAgent (BootstrapAgentStartup): Sending prefix to ddmd for advertisement DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:91 prefix = Ipv6Prefix { addr: fd00:1122:3344:3::, len: 64 }
3872023-12-15T01:09:23.556ZINFOSledAgent (ServiceManager): Writing ledger to /pool/int/b462a7f7-b628-40fe-80ff-4e4189e2d62b/config/.omicron-zones.json.tmp file = common/src/ledger.rs:198
3882023-12-15T01:09:23.556ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 12682388 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::1]:34044 req_id = 9f6e63a5-0510-4182-b908-4ed412dcf84c response_code = 204 uri = /omicron-zones
3892023-12-15T01:09:23.558ZINFOSledAgent (RSS): Configuring DNS server dns_config_addr = [fd00:1122:3344:1::1]:5353 file = sled-agent/src/rack_setup/service.rs:417
3902023-12-15T01:09:23.603ZINFOSledAgent (RSS): Configuring DNS server dns_config_addr = [fd00:1122:3344:2::1]:5353 file = sled-agent/src/rack_setup/service.rs:417
3912023-12-15T01:09:23.652ZINFOSledAgent (RSS): Configuring DNS server dns_config_addr = [fd00:1122:3344:3::1]:5353 file = sled-agent/src/rack_setup/service.rs:417
3922023-12-15T01:09:24.158ZINFOSledAgent (RSS): Configured all DNS servers file = sled-agent/src/rack_setup/service.rs:453
3932023-12-15T01:09:24.158ZINFOSledAgent (RSS): Resolving switch zone addresses in DNS file = sled-agent/src/bootstrap/early_networking.rs:253
3942023-12-15T01:09:24.189ZINFOSledAgent (RSS): Querying MGS to determine switch location addr = fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:270
3952023-12-15T01:09:24.190ZINFOSledAgent (RSS): Found Switch0 at fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:295
3962023-12-15T01:09:24.190ZINFOSledAgent (RSS): Successfully looked up all expected switch zone underlay addresses addrs = {Switch0: fd00:1122:3344:101::2} file = sled-agent/src/bootstrap/early_networking.rs:187
3972023-12-15T01:09:24.220ZINFOSledAgent (RSS): attempting to set up sled's Omicron zones: OmicronZonesConfig { generation: Generation(3), zones: [OmicronZoneConfig { id: c8d17562-2f06-40f3-9824-1f4cb22a7b12, underlay_address: fd00:1122:3344:1::1, zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 14b4dc87-ab46-49fb-a4b4-d361ae214c03, kind: External } }, http_address: [fd00:1122:3344:1::1]:5353, dns_address: [fd00:1122:3344:1::1]:53, gz_address: fd00:1122:3344:1::2, gz_address_index: 0 } }, OmicronZoneConfig { id: aa7234de-3d19-4031-963b-712389d0323e, underlay_address: fd00:1122:3344:2::1, zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 24b4dc87-ab46-49fb-a4b4-d361ae214c03, kind: External } }, http_address: [fd00:1122:3344:2::1]:5353, dns_address: [fd00:1122:3344:2::1]:53, gz_address: fd00:1122:3344:2::2, gz_address_index: 1 } }, OmicronZoneConfig { id: cc39462d-3d1b-41ac-8720-1bae501935df, underlay_address: fd00:1122:3344:3::1, zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 31bd71cd-4736-4a12-a387-9b74b050396f, kind: External } }, http_address: [fd00:1122:3344:3::1]:5353, dns_address: [fd00:1122:3344:3::1]:53, gz_address: fd00:1122:3344:3::2, gz_address_index: 2 } }, OmicronZoneConfig { id: 6e41b21b-bf42-483b-b26f-9cfe96a37942, underlay_address: fd00:1122:3344:101::1b, zone_type: BoundaryNtp { address: [fd00:1122:3344:101::1b]:123, ntp_servers: ["0.pool.ntp.org"], dns_servers: [1.1.1.1, 9.9.9.9], domain: None, nic: NetworkInterface { id: 06202c6c-62fd-4d5b-8f49-b1be87b00df3, kind: Service { id: 6e41b21b-bf42-483b-b26f-9cfe96a37942 }, name: Name("ntp-6e41b21b-bf42-483b-b26f-9cfe96a37942"), ip: 172.30.3.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 218, 40])), subnet: V4(Ipv4Net(Ipv4Network { addr: 172.30.3.0, prefix: 24 })), vni: Vni(100), primary: true, slot: 0 }, snat_cfg: SourceNatConfig { ip: 10.151.2.169, first_port: 0, last_port: 16383 } } }] } file = sled-agent/src/rack_setup/service.rs:299 sled_address = [fd00:1122:3344:101::1]:12345
3982023-12-15T01:09:24.220ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:769 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:36340
3992023-12-15T01:09:24.221ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: c8d17562-2f06-40f3-9824-1f4cb22a7b12, dataset_name: DatasetName { pool_name: ZpoolName { id: 14b4dc87-ab46-49fb-a4b4-d361ae214c03, kind: External }, kind: InternalDns }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } }) file = sled-storage/src/manager.rs:346
4002023-12-15T01:09:24.221ZINFOSledAgent (StorageManager): add_dataset: NewFilesystemRequest { dataset_id: c8d17562-2f06-40f3-9824-1f4cb22a7b12, dataset_name: DatasetName { pool_name: ZpoolName { id: 14b4dc87-ab46-49fb-a4b4-d361ae214c03, kind: External }, kind: InternalDns }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } } file = sled-storage/src/manager.rs:564
4012023-12-15T01:09:24.242ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: aa7234de-3d19-4031-963b-712389d0323e, dataset_name: DatasetName { pool_name: ZpoolName { id: 24b4dc87-ab46-49fb-a4b4-d361ae214c03, kind: External }, kind: InternalDns }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } }) file = sled-storage/src/manager.rs:346
4022023-12-15T01:09:24.245ZINFOSledAgent (StorageManager): add_dataset: NewFilesystemRequest { dataset_id: aa7234de-3d19-4031-963b-712389d0323e, dataset_name: DatasetName { pool_name: ZpoolName { id: 24b4dc87-ab46-49fb-a4b4-d361ae214c03, kind: External }, kind: InternalDns }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } } file = sled-storage/src/manager.rs:564
4032023-12-15T01:09:24.262ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: cc39462d-3d1b-41ac-8720-1bae501935df, dataset_name: DatasetName { pool_name: ZpoolName { id: 31bd71cd-4736-4a12-a387-9b74b050396f, kind: External }, kind: InternalDns }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } }) file = sled-storage/src/manager.rs:346
4042023-12-15T01:09:24.262ZINFOSledAgent (StorageManager): add_dataset: NewFilesystemRequest { dataset_id: cc39462d-3d1b-41ac-8720-1bae501935df, dataset_name: DatasetName { pool_name: ZpoolName { id: 31bd71cd-4736-4a12-a387-9b74b050396f, kind: External }, kind: InternalDns }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } } file = sled-storage/src/manager.rs:564
4052023-12-15T01:09:24.279ZINFOSledAgent (StorageManager): Received GetLatestResources(Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) }) file = sled-storage/src/manager.rs:346
4062023-12-15T01:09:24.279ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/a462a7f7-b628-40fe-80ff-4e4189e2d62b/config/omicron-zones.json file = common/src/ledger.rs:176
4072023-12-15T01:09:24.280ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/b462a7f7-b628-40fe-80ff-4e4189e2d62b/config/omicron-zones.json file = common/src/ledger.rs:176
4082023-12-15T01:09:24.280ZINFOSledAgent (StorageManager): Received GetLatestResources(Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) }) file = sled-storage/src/manager.rs:346
4092023-12-15T01:09:24.282ZINFOSledAgent (ServiceManager): Resolving switch zone addresses in DNS file = sled-agent/src/bootstrap/early_networking.rs:253
4102023-12-15T01:09:24.314ZINFOSledAgent (ServiceManager): Querying MGS to determine switch location addr = fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:270
4112023-12-15T01:09:24.315ZINFOSledAgent (ServiceManager): Found Switch0 at fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:295
4122023-12-15T01:09:24.315ZINFOSledAgent (ServiceManager): Successfully looked up all expected switch zone underlay addresses addrs = {Switch0: fd00:1122:3344:101::2} file = sled-agent/src/bootstrap/early_networking.rs:187
4132023-12-15T01:09:24.359ZINFOSledAgent (PortManager): Created OPTE port file = illumos-utils/src/opte/port_manager.rs:396 port = Port { inner: PortInner { name: "opte0", _ip: 172.30.3.5, mac: MacAddr6([168, 64, 37, 255, 218, 40]), slot: 0, vni: Vni { inner: 100 }, gateway: Gateway { mac: MacAddr6([168, 64, 37, 255, 119, 119]), ip: 172.30.3.1 }, vnic: "vopte0" } }
4142023-12-15T01:09:24.359ZINFOSledAgent (ServiceManager): creating NAT entry for service file = sled-agent/src/services.rs:1221 zone_type = ntp
4152023-12-15T01:09:24.360ZINFOSledAgent (ServiceManager): no nat entry found for: V4( Ipv4Network { addr: 10.151.2.169, prefix: 32, }, ) file = clients/dpd-client/src/lib.rs:98
4162023-12-15T01:09:24.360ZINFOSledAgent (ServiceManager): creating nat entry for: V4( Ipv4Network { addr: 10.151.2.169, prefix: 32, }, ) file = clients/dpd-client/src/lib.rs:105
4172023-12-15T01:09:24.360ZINFOSledAgent (ServiceManager): creation of nat entry successful for: V4( Ipv4Network { addr: 10.151.2.169, prefix: 32, }, ) file = clients/dpd-client/src/lib.rs:133
4182023-12-15T01:09:24.362ZINFOSledAgent (StorageManager): Received GetLatestResources(Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) }) file = sled-storage/src/manager.rs:346
4192023-12-15T01:09:24.385ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_ntp_6e41b21b-bf42-483b-b26f-9cfe96a37942 file = illumos-utils/src/zone.rs:314
4202023-12-15T01:09:24.405ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_ntp_6e41b21b-bf42-483b-b26f-9cfe96a37942 file = illumos-utils/src/zone.rs:353
4212023-12-15T01:09:24.866ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:769 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:60010
4222023-12-15T01:09:24.867ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 63 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:60010 req_id = f0cf22a1-bb7b-455c-999d-ff14fe617a80 response_code = 200 uri = /baseboard
4232023-12-15T01:09:25.417ZINFOSledAgent (ServiceManager): Zone booting file = illumos-utils/src/running_zone.rs:512 zone = oxz_ntp_6e41b21b-bf42-483b-b26f-9cfe96a37942
4242023-12-15T01:09:25.552ZWARNSledAgent (ServiceManager): wait for service Some("oxz_ntp_6e41b21b-bf42-483b-b26f-9cfe96a37942") failed: Property not found. retry in 70.47743ms file = illumos-utils/src/svc.rs:36 zone = oxz_ntp_6e41b21b-bf42-483b-b26f-9cfe96a37942
4252023-12-15T01:09:25.637ZWARNSledAgent (ServiceManager): wait for service Some("oxz_ntp_6e41b21b-bf42-483b-b26f-9cfe96a37942") failed: Property not found. retry in 107.608945ms file = illumos-utils/src/svc.rs:36 zone = oxz_ntp_6e41b21b-bf42-483b-b26f-9cfe96a37942
4262023-12-15T01:09:25.782ZWARNSledAgent (ServiceManager): wait for service Some("oxz_ntp_6e41b21b-bf42-483b-b26f-9cfe96a37942") failed: Property not found. retry in 197.686785ms file = illumos-utils/src/svc.rs:36 zone = oxz_ntp_6e41b21b-bf42-483b-b26f-9cfe96a37942
4272023-12-15T01:09:25.992ZWARNSledAgent (ServiceManager): wait for service Some("oxz_ntp_6e41b21b-bf42-483b-b26f-9cfe96a37942") failed: Property not found. retry in 286.299463ms file = illumos-utils/src/svc.rs:36 zone = oxz_ntp_6e41b21b-bf42-483b-b26f-9cfe96a37942
4282023-12-15T01:09:26.287ZWARNSledAgent (ServiceManager): wait for service Some("oxz_ntp_6e41b21b-bf42-483b-b26f-9cfe96a37942") failed: Property not found. retry in 965.735363ms file = illumos-utils/src/svc.rs:36 zone = oxz_ntp_6e41b21b-bf42-483b-b26f-9cfe96a37942
4292023-12-15T01:09:27.262ZWARNSledAgent (ServiceManager): wait for service Some("oxz_ntp_6e41b21b-bf42-483b-b26f-9cfe96a37942") failed: Property not found. retry in 1.164110179s file = illumos-utils/src/svc.rs:36 zone = oxz_ntp_6e41b21b-bf42-483b-b26f-9cfe96a37942
4302023-12-15T01:09:27.877ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::605a:15ff:fea0:c066": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
4312023-12-15T01:09:28.434ZWARNSledAgent (ServiceManager): wait for service Some("oxz_ntp_6e41b21b-bf42-483b-b26f-9cfe96a37942") failed: Property not found. retry in 1.453852216s file = illumos-utils/src/svc.rs:36 zone = oxz_ntp_6e41b21b-bf42-483b-b26f-9cfe96a37942
4322023-12-15T01:09:29.897ZWARNSledAgent (ServiceManager): wait for service Some("oxz_ntp_6e41b21b-bf42-483b-b26f-9cfe96a37942") failed: Property not found. retry in 1.428327314s file = illumos-utils/src/svc.rs:36 zone = oxz_ntp_6e41b21b-bf42-483b-b26f-9cfe96a37942
4332023-12-15T01:09:31.366ZINFOSledAgent (ServiceManager): Ensuring address fd00:1122:3344:101::1b exists file = sled-agent/src/services.rs:1764
4342023-12-15T01:09:31.366ZINFOSledAgent (ServiceManager): Adding address: Static(V6(Ipv6Network { addr: fd00:1122:3344:101::1b, prefix: 64 })) file = illumos-utils/src/running_zone.rs:608 zone = oxz_ntp_6e41b21b-bf42-483b-b26f-9cfe96a37942
4352023-12-15T01:09:31.917ZINFOSledAgent (ServiceManager): Ensuring address fd00:1122:3344:101::1b exists - OK file = sled-agent/src/services.rs:1772
4362023-12-15T01:09:31.917ZINFOSledAgent (ServiceManager): Zone using sled underlay as gateway file = sled-agent/src/services.rs:1791
4372023-12-15T01:09:32.231ZINFOSledAgent (ServiceManager): Set up NTP service boundary=true, Servers=["0.pool.ntp.org"] file = sled-agent/src/services.rs:2081
4382023-12-15T01:09:32.265ZINFOSledAgent (ServiceManager): Ensuring address for OPTE port file = illumos-utils/src/running_zone.rs:652 zone = oxz_ntp_6e41b21b-bf42-483b-b26f-9cfe96a37942
4392023-12-15T01:09:36.892ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::605a:15ff:fea0:c066": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
4402023-12-15T01:09:36.948ZINFOSledAgent (ServiceManager): Writing ledger to /pool/int/a462a7f7-b628-40fe-80ff-4e4189e2d62b/config/.omicron-zones.json.tmp file = common/src/ledger.rs:198
4412023-12-15T01:09:36.948ZINFOSledAgent (ServiceManager): Writing ledger to /pool/int/b462a7f7-b628-40fe-80ff-4e4189e2d62b/config/.omicron-zones.json.tmp file = common/src/ledger.rs:198
4422023-12-15T01:09:36.948ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 12727618 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::1]:36340 req_id = 17fff5a3-9d8d-41c5-a27f-feb667ad44d3 response_code = 204 uri = /omicron-zones
4432023-12-15T01:09:36.951ZINFOSledAgent (RSS): Waiting for rack time synchronization file = sled-agent/src/rack_setup/service.rs:494
4442023-12-15T01:09:36.981ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:474
4452023-12-15T01:09:36.981ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:769 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:55279
4462023-12-15T01:09:36.992ZINFOSledAgent (ServiceManager): chronyc command failed: Error running command in zone 'oxz_ntp_6e41b21b-bf42-483b-b26f-9cfe96a37942': Command [/usr/bin/chronyc -c tracking] executed and failed with status: exit status: 1 stdout: 506 Cannot talk to daemon stderr: file = sled-agent/src/services.rs:3023
4472023-12-15T01:09:36.992ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = NTP zone not ready file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:841 latency_us = 10427 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:55279 req_id = 7559b8a3-8315-4cd3-a620-1b3e93bff2a0 response_code = 500 uri = /timesync
4482023-12-15T01:09:36.994ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:524
4492023-12-15T01:09:37.080ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:474
4502023-12-15T01:09:37.080ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:769 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:33514
4512023-12-15T01:09:37.088ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 7753 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:33514 req_id = abbb6eb8-adf6-4a8e-abfb-33a5cd17a4b9 response_code = 200 uri = /timesync
4522023-12-15T01:09:37.090ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 2139029761, ip_addr: ::, stratum: 10, ref_time: 1702602575.7445965, correction: 0.0 } file = sled-agent/src/rack_setup/service.rs:502
4532023-12-15T01:09:37.090ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:524
4542023-12-15T01:09:37.252ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:474
4552023-12-15T01:09:37.252ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:769 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:54604
4562023-12-15T01:09:37.261ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 8240 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:54604 req_id = 3b36b66e-30a4-4cbc-8ec9-848ec0082919 response_code = 200 uri = /timesync
4572023-12-15T01:09:37.262ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 2139029761, ip_addr: ::, stratum: 10, ref_time: 1702602575.7445965, correction: 0.0 } file = sled-agent/src/rack_setup/service.rs:502
4582023-12-15T01:09:37.262ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:524
4592023-12-15T01:09:37.494ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:474
4602023-12-15T01:09:37.494ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:769 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:57241
4612023-12-15T01:09:37.502ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 7871 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:57241 req_id = c3dcda0c-20cc-4d2f-8e30-48dc0b2ce0ba response_code = 200 uri = /timesync
4622023-12-15T01:09:37.504ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 2139029761, ip_addr: ::, stratum: 10, ref_time: 1702602575.7445965, correction: 0.0 } file = sled-agent/src/rack_setup/service.rs:502
4632023-12-15T01:09:37.504ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:524
4642023-12-15T01:09:37.688ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:474
4652023-12-15T01:09:37.688ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:769 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:62870
4662023-12-15T01:09:37.696ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 8329 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:62870 req_id = 4bb681cd-c0a9-4f0b-9ceb-3a817930dbe9 response_code = 200 uri = /timesync
4672023-12-15T01:09:37.698ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 2139029761, ip_addr: ::, stratum: 10, ref_time: 1702602575.7445965, correction: 0.0 } file = sled-agent/src/rack_setup/service.rs:502
4682023-12-15T01:09:37.698ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:524
4692023-12-15T01:09:37.852ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:474
4702023-12-15T01:09:37.852ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:769 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:44722
4712023-12-15T01:09:37.860ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 8122 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:44722 req_id = afa82391-b5aa-4f62-9a4a-77e5bbc0da5f response_code = 200 uri = /timesync
4722023-12-15T01:09:37.862ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 2139029761, ip_addr: ::, stratum: 10, ref_time: 1702602575.7445965, correction: 0.0 } file = sled-agent/src/rack_setup/service.rs:502
4732023-12-15T01:09:37.862ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:524
4742023-12-15T01:09:38.036ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:474
4752023-12-15T01:09:38.037ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:769 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:55122
4762023-12-15T01:09:38.045ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 8191 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:55122 req_id = eac2d532-4509-4eb9-ad4c-4627074d2118 response_code = 200 uri = /timesync
4772023-12-15T01:09:38.046ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 2139029761, ip_addr: ::, stratum: 10, ref_time: 1702602575.7445965, correction: 0.0 } file = sled-agent/src/rack_setup/service.rs:502
4782023-12-15T01:09:38.046ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:524
4792023-12-15T01:09:38.448ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:474
4802023-12-15T01:09:38.448ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:769 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:43955
4812023-12-15T01:09:38.456ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 7988 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:43955 req_id = be2ccd3d-de2e-444a-82cf-05772f52af21 response_code = 200 uri = /timesync
4822023-12-15T01:09:38.458ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 2139029761, ip_addr: ::, stratum: 10, ref_time: 1702602575.7445965, correction: 0.0 } file = sled-agent/src/rack_setup/service.rs:502
4832023-12-15T01:09:38.458ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:524
4842023-12-15T01:09:38.769ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:474
4852023-12-15T01:09:38.769ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:769 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:51200
4862023-12-15T01:09:38.777ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 8598 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:51200 req_id = c7b5c062-95d0-4c1c-a122-a449ed2da70a response_code = 200 uri = /timesync
4872023-12-15T01:09:38.779ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 2139029761, ip_addr: ::, stratum: 10, ref_time: 1702602575.7445965, correction: 0.0 } file = sled-agent/src/rack_setup/service.rs:502
4882023-12-15T01:09:38.779ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:524
4892023-12-15T01:09:39.197ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:474
4902023-12-15T01:09:39.197ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:769 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:49323
4912023-12-15T01:09:39.206ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 8224 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:49323 req_id = 171c3b80-fce9-4120-af06-31ba0575ad48 response_code = 200 uri = /timesync
4922023-12-15T01:09:39.207ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 2139029761, ip_addr: ::, stratum: 10, ref_time: 1702602575.7445965, correction: 0.0 } file = sled-agent/src/rack_setup/service.rs:502
4932023-12-15T01:09:39.207ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:524
4942023-12-15T01:09:39.505ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:474
4952023-12-15T01:09:39.505ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:769 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:61982
4962023-12-15T01:09:39.513ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 7859 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:61982 req_id = fb27bc81-064c-4ada-970f-04ce0cceb8bc response_code = 200 uri = /timesync
4972023-12-15T01:09:39.515ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 2139029761, ip_addr: ::, stratum: 10, ref_time: 1702602575.7445965, correction: 0.0 } file = sled-agent/src/rack_setup/service.rs:502
4982023-12-15T01:09:39.515ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:524
4992023-12-15T01:09:40.098ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:474
5002023-12-15T01:09:40.099ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:769 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:33288
5012023-12-15T01:09:40.107ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 8251 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:33288 req_id = 2e0044fc-4ba6-4d6f-ae4a-d71f37746c7e response_code = 200 uri = /timesync
5022023-12-15T01:09:40.110ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 2139029761, ip_addr: ::, stratum: 10, ref_time: 1702602575.7445965, correction: 0.0 } file = sled-agent/src/rack_setup/service.rs:502
5032023-12-15T01:09:40.110ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:524
5042023-12-15T01:09:40.605ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:474
5052023-12-15T01:09:40.605ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:769 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:57653
5062023-12-15T01:09:40.614ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 8910 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:57653 req_id = e5492459-4dc8-413c-ae94-78fe5a4ab0bb response_code = 200 uri = /timesync
5072023-12-15T01:09:40.616ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 2139029761, ip_addr: ::, stratum: 10, ref_time: 1702602575.7445965, correction: 0.0 } file = sled-agent/src/rack_setup/service.rs:502
5082023-12-15T01:09:40.616ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:524
5092023-12-15T01:09:41.864ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:474
5102023-12-15T01:09:41.864ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:769 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:46951
5112023-12-15T01:09:41.873ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 8503 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:46951 req_id = 41e8cea9-6007-4664-8a64-136b5eab3f61 response_code = 200 uri = /timesync
5122023-12-15T01:09:41.875ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 2139029761, ip_addr: ::, stratum: 10, ref_time: 1702602575.7445965, correction: 0.0 } file = sled-agent/src/rack_setup/service.rs:502
5132023-12-15T01:09:41.875ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:524
5142023-12-15T01:09:41.894ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::605a:15ff:fea0:c066": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
5152023-12-15T01:09:43.146ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:474
5162023-12-15T01:09:43.146ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:769 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:49387
5172023-12-15T01:09:43.149ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:532
5182023-12-15T01:09:43.155ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 8900 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:49387 req_id = 4a20ff2b-ccae-43da-991a-4fe8dd8241ea response_code = 200 uri = /timesync
5192023-12-15T01:09:43.158ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 2139029761, ip_addr: ::, stratum: 10, ref_time: 1702602575.7445965, correction: 0.0 } file = sled-agent/src/rack_setup/service.rs:502
5202023-12-15T01:09:43.158ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:524
5212023-12-15T01:09:44.248ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:474
5222023-12-15T01:09:44.248ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:769 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:65225
5232023-12-15T01:09:44.252ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:532
5242023-12-15T01:09:44.257ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 8670 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:65225 req_id = f933b9e3-ef86-4268-83fe-669a6750d57f response_code = 200 uri = /timesync
5252023-12-15T01:09:44.259ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 2310930948, ip_addr: 137.190.2.4, stratum: 2, ref_time: 1702602583.0351586, correction: -0.291977704 } file = sled-agent/src/rack_setup/service.rs:502
5262023-12-15T01:09:44.259ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:524
5272023-12-15T01:09:45.430ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:474
5282023-12-15T01:09:45.430ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:769 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:36067
5292023-12-15T01:09:45.439ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 8651 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:36067 req_id = 0155edd4-7e76-4ec3-9985-f4eb5be231f5 response_code = 200 uri = /timesync
5302023-12-15T01:09:45.441ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 2139029761, ip_addr: ::, stratum: 10, ref_time: 1702602575.7445965, correction: -0.288768291 } file = sled-agent/src/rack_setup/service.rs:502
5312023-12-15T01:09:45.441ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:524
5322023-12-15T01:09:46.773ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:474
5332023-12-15T01:09:46.773ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:769 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:45079
5342023-12-15T01:09:46.782ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 8134 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:45079 req_id = 91428b34-d788-4f3e-97a5-699d4ea9ae2c response_code = 200 uri = /timesync
5352023-12-15T01:09:46.784ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 2139029761, ip_addr: ::, stratum: 10, ref_time: 1702602575.7445965, correction: -0.285121948 } file = sled-agent/src/rack_setup/service.rs:502
5362023-12-15T01:09:46.784ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:524
5372023-12-15T01:09:47.035ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::605a:15ff:fea0:c066": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
5382023-12-15T01:09:48.346ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:474
5392023-12-15T01:09:48.346ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:769 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:57394
5402023-12-15T01:09:48.354ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 8288 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:57394 req_id = 33c4d5ed-873b-4973-ba2f-c4c54e1991d6 response_code = 200 uri = /timesync
5412023-12-15T01:09:48.356ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 2139029761, ip_addr: ::, stratum: 10, ref_time: 1702602575.7445965, correction: -0.280851871 } file = sled-agent/src/rack_setup/service.rs:502
5422023-12-15T01:09:48.356ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:524
5432023-12-15T01:09:52.023ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::605a:15ff:fea0:c066": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
5442023-12-15T01:09:52.313ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:474
5452023-12-15T01:09:52.313ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:769 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:54709
5462023-12-15T01:09:52.322ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 9107 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:54709 req_id = 9736c8cd-94f2-44c7-940a-ec1c6cbb118b response_code = 200 uri = /timesync
5472023-12-15T01:09:52.324ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 2139029761, ip_addr: ::, stratum: 10, ref_time: 1702602575.7445965, correction: -0.270076007 } file = sled-agent/src/rack_setup/service.rs:502
5482023-12-15T01:09:52.324ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:524
5492023-12-15T01:09:55.020ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:769 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:50614
5502023-12-15T01:09:55.020ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 56 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:50614 req_id = 4d0da698-9d38-4b59-bd37-d06798991d88 response_code = 200 uri = /baseboard
5512023-12-15T01:09:55.185ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:474
5522023-12-15T01:09:55.185ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:769 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:63945
5532023-12-15T01:09:55.194ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 8495 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:63945 req_id = df31b075-0ce6-433c-b25d-b50f0d95e46f response_code = 200 uri = /timesync
5542023-12-15T01:09:55.196ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 2139029761, ip_addr: ::, stratum: 10, ref_time: 1702602575.7445965, correction: -0.262277305 } file = sled-agent/src/rack_setup/service.rs:502
5552023-12-15T01:09:55.196ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:524
5562023-12-15T01:09:57.011ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::605a:15ff:fea0:c066": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
5572023-12-15T01:09:59.773ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:474
5582023-12-15T01:09:59.773ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:769 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:62081
5592023-12-15T01:09:59.782ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 8812 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:62081 req_id = 0fc26332-208b-4922-aa49-3d65add599e6 response_code = 200 uri = /timesync
5602023-12-15T01:09:59.785ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 2139029761, ip_addr: ::, stratum: 10, ref_time: 1702602575.7445965, correction: -0.249818444 } file = sled-agent/src/rack_setup/service.rs:502
5612023-12-15T01:09:59.785ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:524
5622023-12-15T01:10:02.000ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::605a:15ff:fea0:c066": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
5632023-12-15T01:10:03.366ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:474
5642023-12-15T01:10:03.366ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:769 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:35452
5652023-12-15T01:10:03.374ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 8243 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:35452 req_id = ce9dccb1-7d7b-46ed-9871-9a813f9cc62f response_code = 200 uri = /timesync
5662023-12-15T01:10:03.376ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 2139029761, ip_addr: ::, stratum: 10, ref_time: 1702602575.7445965, correction: -0.240061164 } file = sled-agent/src/rack_setup/service.rs:502
5672023-12-15T01:10:03.376ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:524
5682023-12-15T01:10:06.988ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::605a:15ff:fea0:c066": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
5692023-12-15T01:10:07.055ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:474
5702023-12-15T01:10:07.056ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:769 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:51153
5712023-12-15T01:10:07.064ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 8839 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:51153 req_id = b5e1a77a-6d29-48a4-8ed1-5041ebddfc5f response_code = 200 uri = /timesync
5722023-12-15T01:10:07.066ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 2139029761, ip_addr: ::, stratum: 10, ref_time: 1702602575.7445965, correction: -0.230040312 } file = sled-agent/src/rack_setup/service.rs:502
5732023-12-15T01:10:07.066ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:524
5742023-12-15T01:10:11.975ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::605a:15ff:fea0:c066": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
5752023-12-15T01:10:12.275ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:474
5762023-12-15T01:10:12.275ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:769 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:44616
5772023-12-15T01:10:12.283ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 8099 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:44616 req_id = 3c159c0a-f3e7-4fe3-b886-a71630547f13 response_code = 200 uri = /timesync
5782023-12-15T01:10:12.285ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 2139029761, ip_addr: ::, stratum: 10, ref_time: 1702602575.7445965, correction: -0.215868503 } file = sled-agent/src/rack_setup/service.rs:502
5792023-12-15T01:10:12.285ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:524
5802023-12-15T01:10:16.963ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::605a:15ff:fea0:c066": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
5812023-12-15T01:10:21.309ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:474
5822023-12-15T01:10:21.309ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:769 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:41708
5832023-12-15T01:10:21.318ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 8718 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:41708 req_id = 76daa4ce-ac11-4f22-b8b1-a4ee72c94160 response_code = 200 uri = /timesync
5842023-12-15T01:10:21.320ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 2139029761, ip_addr: ::, stratum: 10, ref_time: 1702602575.7445965, correction: -0.191332921 } file = sled-agent/src/rack_setup/service.rs:502
5852023-12-15T01:10:21.320ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:524
5862023-12-15T01:10:21.951ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::605a:15ff:fea0:c066": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
5872023-12-15T01:10:24.972ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:769 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:56887
5882023-12-15T01:10:24.973ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 746 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:56887 req_id = 9524d6e5-9edd-4908-a8c7-56f47001d733 response_code = 200 uri = /baseboard
5892023-12-15T01:10:26.939ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::605a:15ff:fea0:c066": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
5902023-12-15T01:10:28.928ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:474
5912023-12-15T01:10:28.928ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:769 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:60743
5922023-12-15T01:10:28.937ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 8364 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:60743 req_id = 497a0924-04bb-404b-8415-2382028338cd response_code = 200 uri = /timesync
5932023-12-15T01:10:28.939ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 2139029761, ip_addr: ::, stratum: 10, ref_time: 1702602575.7445965, correction: -0.170642316 } file = sled-agent/src/rack_setup/service.rs:502
5942023-12-15T01:10:28.939ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:524
5952023-12-15T01:10:31.927ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::605a:15ff:fea0:c066": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
5962023-12-15T01:10:36.915ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::605a:15ff:fea0:c066": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
5972023-12-15T01:10:38.734ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:474
5982023-12-15T01:10:38.734ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:769 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:52863
5992023-12-15T01:10:38.743ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 8636 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:52863 req_id = af3e5d33-d634-4bfb-b71f-b9d43b54f3af response_code = 200 uri = /timesync
6002023-12-15T01:10:38.745ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 2139029761, ip_addr: ::, stratum: 10, ref_time: 1702602575.7445965, correction: -0.144011617 } file = sled-agent/src/rack_setup/service.rs:502
6012023-12-15T01:10:38.745ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:524
6022023-12-15T01:10:41.903ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::605a:15ff:fea0:c066": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
6032023-12-15T01:10:46.890ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::605a:15ff:fea0:c066": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
6042023-12-15T01:10:47.065ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:474
6052023-12-15T01:10:47.065ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:769 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:32852
6062023-12-15T01:10:47.074ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 8615 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:32852 req_id = e662c590-fd01-4f80-843a-fec181be065d response_code = 200 uri = /timesync
6072023-12-15T01:10:47.076ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 2139029761, ip_addr: ::, stratum: 10, ref_time: 1702602645.2586336, correction: -0.121388003 } file = sled-agent/src/rack_setup/service.rs:502
6082023-12-15T01:10:47.076ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:524
6092023-12-15T01:10:51.879ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::605a:15ff:fea0:c066": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
6102023-12-15T01:10:54.924ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:769 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:47146
6112023-12-15T01:10:54.924ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 49 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:47146 req_id = 28288cc9-58b4-488c-9e2b-608b768ebf99 response_code = 200 uri = /baseboard
6122023-12-15T01:10:55.629ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:474
6132023-12-15T01:10:55.629ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:769 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:56317
6142023-12-15T01:10:55.639ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 9471 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:56317 req_id = 1ba4769c-7648-4472-a3e3-b1978ebf33b7 response_code = 200 uri = /timesync
6152023-12-15T01:10:55.641ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 2139029761, ip_addr: ::, stratum: 10, ref_time: 1702602645.2586336, correction: -0.102852203 } file = sled-agent/src/rack_setup/service.rs:502
6162023-12-15T01:10:55.641ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:524
6172023-12-15T01:10:56.867ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::605a:15ff:fea0:c066": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
6182023-12-15T01:11:01.856ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::605a:15ff:fea0:c066": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
6192023-12-15T01:11:06.844ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::605a:15ff:fea0:c066": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
6202023-12-15T01:11:11.832ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::605a:15ff:fea0:c066": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
6212023-12-15T01:11:16.557ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:474
6222023-12-15T01:11:16.557ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:769 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:62705
6232023-12-15T01:11:16.566ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 8840 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:62705 req_id = 4590c3a7-a75b-4637-bcda-081b1a1e6017 response_code = 200 uri = /timesync
6242023-12-15T01:11:16.568ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 2139029761, ip_addr: ::, stratum: 10, ref_time: 1702602645.2586336, correction: -0.046019826 } file = sled-agent/src/rack_setup/service.rs:502
6252023-12-15T01:11:16.568ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:524
6262023-12-15T01:11:16.820ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::605a:15ff:fea0:c066": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
6272023-12-15T01:11:21.808ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::605a:15ff:fea0:c066": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
6282023-12-15T01:11:24.876ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:769 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:49729
6292023-12-15T01:11:24.876ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 65 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:49729 req_id = 53e5dcfb-8207-465c-8bbd-ebe568fd0cde response_code = 200 uri = /baseboard
6302023-12-15T01:11:26.795ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::605a:15ff:fea0:c066": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
6312023-12-15T01:11:30.446ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:474
6322023-12-15T01:11:30.446ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:769 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:38676
6332023-12-15T01:11:30.455ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 9260 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:38676 req_id = 0dca4d7e-87c0-4b1b-bec6-54e0d8ca7070 response_code = 200 uri = /timesync
6342023-12-15T01:11:30.457ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 2139029761, ip_addr: ::, stratum: 10, ref_time: 1702602645.2586336, correction: -0.008301036 } file = sled-agent/src/rack_setup/service.rs:502
6352023-12-15T01:11:30.457ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:524
6362023-12-15T01:11:31.783ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::605a:15ff:fea0:c066": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
6372023-12-15T01:11:36.778ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::605a:15ff:fea0:c066": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
6382023-12-15T01:11:41.780ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::605a:15ff:fea0:c066": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
6392023-12-15T01:11:46.781ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::605a:15ff:fea0:c066": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
6402023-12-15T01:11:51.782ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::605a:15ff:fea0:c066": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
6412023-12-15T01:11:54.887ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:769 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:39934
6422023-12-15T01:11:54.887ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 56 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:39934 req_id = a9d55b50-b330-4a1a-84eb-fbf5599b075c response_code = 200 uri = /baseboard
6432023-12-15T01:11:56.784ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::605a:15ff:fea0:c066": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
6442023-12-15T01:12:01.787ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::605a:15ff:fea0:c066": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
6452023-12-15T01:12:01.864ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:474
6462023-12-15T01:12:01.864ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:769 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:55378
6472023-12-15T01:12:01.889ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 24717 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:55378 req_id = 284efc30-119d-46b5-9a1e-e34e604d92cc response_code = 200 uri = /timesync
6482023-12-15T01:12:01.891ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: true, ref_id: 2728380417, ip_addr: 162.159.200.1, stratum: 4, ref_time: 1702602711.5197906, correction: 0.000924246 } file = sled-agent/src/rack_setup/service.rs:502
6492023-12-15T01:12:01.891ZINFOSledAgent (RSS): Finished setting up Internal DNS and NTP file = sled-agent/src/rack_setup/service.rs:1042
6502023-12-15T01:12:01.920ZINFOSledAgent (RSS): attempting to set up sled's Omicron zones: OmicronZonesConfig { generation: Generation(4), zones: [OmicronZoneConfig { id: c8d17562-2f06-40f3-9824-1f4cb22a7b12, underlay_address: fd00:1122:3344:1::1, zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 14b4dc87-ab46-49fb-a4b4-d361ae214c03, kind: External } }, http_address: [fd00:1122:3344:1::1]:5353, dns_address: [fd00:1122:3344:1::1]:53, gz_address: fd00:1122:3344:1::2, gz_address_index: 0 } }, OmicronZoneConfig { id: aa7234de-3d19-4031-963b-712389d0323e, underlay_address: fd00:1122:3344:2::1, zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 24b4dc87-ab46-49fb-a4b4-d361ae214c03, kind: External } }, http_address: [fd00:1122:3344:2::1]:5353, dns_address: [fd00:1122:3344:2::1]:53, gz_address: fd00:1122:3344:2::2, gz_address_index: 1 } }, OmicronZoneConfig { id: cc39462d-3d1b-41ac-8720-1bae501935df, underlay_address: fd00:1122:3344:3::1, zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 31bd71cd-4736-4a12-a387-9b74b050396f, kind: External } }, http_address: [fd00:1122:3344:3::1]:5353, dns_address: [fd00:1122:3344:3::1]:53, gz_address: fd00:1122:3344:3::2, gz_address_index: 2 } }, OmicronZoneConfig { id: 6e41b21b-bf42-483b-b26f-9cfe96a37942, underlay_address: fd00:1122:3344:101::1b, zone_type: BoundaryNtp { address: [fd00:1122:3344:101::1b]:123, ntp_servers: ["0.pool.ntp.org"], dns_servers: [1.1.1.1, 9.9.9.9], domain: None, nic: NetworkInterface { id: 06202c6c-62fd-4d5b-8f49-b1be87b00df3, kind: Service { id: 6e41b21b-bf42-483b-b26f-9cfe96a37942 }, name: Name("ntp-6e41b21b-bf42-483b-b26f-9cfe96a37942"), ip: 172.30.3.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 218, 40])), subnet: V4(Ipv4Net(Ipv4Network { addr: 172.30.3.0, prefix: 24 })), vni: Vni(100), primary: true, slot: 0 }, snat_cfg: SourceNatConfig { ip: 10.151.2.169, first_port: 0, last_port: 16383 } } }, OmicronZoneConfig { id: b5d8727f-e19d-4672-a2f8-930bec445061, underlay_address: fd00:1122:3344:101::3, zone_type: CockroachDb { address: [fd00:1122:3344:101::3]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 14b4dc87-ab46-49fb-a4b4-d361ae214c03, kind: External } } } }, OmicronZoneConfig { id: 9f6904ab-71f4-4e02-8b40-0deb1fbe77aa, underlay_address: fd00:1122:3344:101::4, zone_type: CockroachDb { address: [fd00:1122:3344:101::4]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 24b4dc87-ab46-49fb-a4b4-d361ae214c03, kind: External } } } }, OmicronZoneConfig { id: c80d7f41-e0c5-46c6-b353-df45d1040963, underlay_address: fd00:1122:3344:101::5, zone_type: CockroachDb { address: [fd00:1122:3344:101::5]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 31bd71cd-4736-4a12-a387-9b74b050396f, kind: External } } } }, OmicronZoneConfig { id: 00da64ed-baa0-403b-a869-43a61433bbd9, underlay_address: fd00:1122:3344:101::6, zone_type: CockroachDb { address: [fd00:1122:3344:101::6]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 616b26df-e62a-4c68-b506-f4a923d8aaf7, kind: External } } } }, OmicronZoneConfig { id: 7fc3d62c-c7ff-41aa-be0f-d678bc85f00a, underlay_address: fd00:1122:3344:101::7, zone_type: CockroachDb { address: [fd00:1122:3344:101::7]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: cd70d7f6-2354-4bf2-8012-55bf9eaf7930, kind: External } } } }] } file = sled-agent/src/rack_setup/service.rs:299 sled_address = [fd00:1122:3344:101::1]:12345
6512023-12-15T01:12:01.921ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:769 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:55091
6522023-12-15T01:12:01.921ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: c8d17562-2f06-40f3-9824-1f4cb22a7b12, dataset_name: DatasetName { pool_name: ZpoolName { id: 14b4dc87-ab46-49fb-a4b4-d361ae214c03, kind: External }, kind: InternalDns }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } }) file = sled-storage/src/manager.rs:346
6532023-12-15T01:12:01.921ZINFOSledAgent (StorageManager): add_dataset: NewFilesystemRequest { dataset_id: c8d17562-2f06-40f3-9824-1f4cb22a7b12, dataset_name: DatasetName { pool_name: ZpoolName { id: 14b4dc87-ab46-49fb-a4b4-d361ae214c03, kind: External }, kind: InternalDns }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } } file = sled-storage/src/manager.rs:564
6542023-12-15T01:12:01.942ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: aa7234de-3d19-4031-963b-712389d0323e, dataset_name: DatasetName { pool_name: ZpoolName { id: 24b4dc87-ab46-49fb-a4b4-d361ae214c03, kind: External }, kind: InternalDns }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } }) file = sled-storage/src/manager.rs:346
6552023-12-15T01:12:01.942ZINFOSledAgent (StorageManager): add_dataset: NewFilesystemRequest { dataset_id: aa7234de-3d19-4031-963b-712389d0323e, dataset_name: DatasetName { pool_name: ZpoolName { id: 24b4dc87-ab46-49fb-a4b4-d361ae214c03, kind: External }, kind: InternalDns }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } } file = sled-storage/src/manager.rs:564
6562023-12-15T01:12:01.960ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: cc39462d-3d1b-41ac-8720-1bae501935df, dataset_name: DatasetName { pool_name: ZpoolName { id: 31bd71cd-4736-4a12-a387-9b74b050396f, kind: External }, kind: InternalDns }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } }) file = sled-storage/src/manager.rs:346
6572023-12-15T01:12:01.960ZINFOSledAgent (StorageManager): add_dataset: NewFilesystemRequest { dataset_id: cc39462d-3d1b-41ac-8720-1bae501935df, dataset_name: DatasetName { pool_name: ZpoolName { id: 31bd71cd-4736-4a12-a387-9b74b050396f, kind: External }, kind: InternalDns }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } } file = sled-storage/src/manager.rs:564
6582023-12-15T01:12:01.980ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: b5d8727f-e19d-4672-a2f8-930bec445061, dataset_name: DatasetName { pool_name: ZpoolName { id: 14b4dc87-ab46-49fb-a4b4-d361ae214c03, kind: External }, kind: CockroachDb }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } }) file = sled-storage/src/manager.rs:346
6592023-12-15T01:12:01.980ZINFOSledAgent (StorageManager): add_dataset: NewFilesystemRequest { dataset_id: b5d8727f-e19d-4672-a2f8-930bec445061, dataset_name: DatasetName { pool_name: ZpoolName { id: 14b4dc87-ab46-49fb-a4b4-d361ae214c03, kind: External }, kind: CockroachDb }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } } file = sled-storage/src/manager.rs:564
6602023-12-15T01:12:02.045ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 9f6904ab-71f4-4e02-8b40-0deb1fbe77aa, dataset_name: DatasetName { pool_name: ZpoolName { id: 24b4dc87-ab46-49fb-a4b4-d361ae214c03, kind: External }, kind: CockroachDb }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } }) file = sled-storage/src/manager.rs:346
6612023-12-15T01:12:02.045ZINFOSledAgent (StorageManager): add_dataset: NewFilesystemRequest { dataset_id: 9f6904ab-71f4-4e02-8b40-0deb1fbe77aa, dataset_name: DatasetName { pool_name: ZpoolName { id: 24b4dc87-ab46-49fb-a4b4-d361ae214c03, kind: External }, kind: CockroachDb }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } } file = sled-storage/src/manager.rs:564
6622023-12-15T01:12:02.101ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: c80d7f41-e0c5-46c6-b353-df45d1040963, dataset_name: DatasetName { pool_name: ZpoolName { id: 31bd71cd-4736-4a12-a387-9b74b050396f, kind: External }, kind: CockroachDb }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } }) file = sled-storage/src/manager.rs:346
6632023-12-15T01:12:02.101ZINFOSledAgent (StorageManager): add_dataset: NewFilesystemRequest { dataset_id: c80d7f41-e0c5-46c6-b353-df45d1040963, dataset_name: DatasetName { pool_name: ZpoolName { id: 31bd71cd-4736-4a12-a387-9b74b050396f, kind: External }, kind: CockroachDb }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } } file = sled-storage/src/manager.rs:564
6642023-12-15T01:12:02.153ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 00da64ed-baa0-403b-a869-43a61433bbd9, dataset_name: DatasetName { pool_name: ZpoolName { id: 616b26df-e62a-4c68-b506-f4a923d8aaf7, kind: External }, kind: CockroachDb }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } }) file = sled-storage/src/manager.rs:346
6652023-12-15T01:12:02.153ZINFOSledAgent (StorageManager): add_dataset: NewFilesystemRequest { dataset_id: 00da64ed-baa0-403b-a869-43a61433bbd9, dataset_name: DatasetName { pool_name: ZpoolName { id: 616b26df-e62a-4c68-b506-f4a923d8aaf7, kind: External }, kind: CockroachDb }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } } file = sled-storage/src/manager.rs:564
6662023-12-15T01:12:02.208ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 7fc3d62c-c7ff-41aa-be0f-d678bc85f00a, dataset_name: DatasetName { pool_name: ZpoolName { id: cd70d7f6-2354-4bf2-8012-55bf9eaf7930, kind: External }, kind: CockroachDb }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } }) file = sled-storage/src/manager.rs:346
6672023-12-15T01:12:02.208ZINFOSledAgent (StorageManager): add_dataset: NewFilesystemRequest { dataset_id: 7fc3d62c-c7ff-41aa-be0f-d678bc85f00a, dataset_name: DatasetName { pool_name: ZpoolName { id: cd70d7f6-2354-4bf2-8012-55bf9eaf7930, kind: External }, kind: CockroachDb }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } } file = sled-storage/src/manager.rs:564
6682023-12-15T01:12:02.263ZINFOSledAgent (StorageManager): Received GetLatestResources(Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) }) file = sled-storage/src/manager.rs:346
6692023-12-15T01:12:02.263ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/a462a7f7-b628-40fe-80ff-4e4189e2d62b/config/omicron-zones.json file = common/src/ledger.rs:176
6702023-12-15T01:12:02.264ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/b462a7f7-b628-40fe-80ff-4e4189e2d62b/config/omicron-zones.json file = common/src/ledger.rs:176
6712023-12-15T01:12:02.264ZINFOSledAgent (StorageManager): Received GetLatestResources(Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) }) file = sled-storage/src/manager.rs:346
6722023-12-15T01:12:02.271ZINFOSledAgent (StorageManager): Received GetLatestResources(Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) }) file = sled-storage/src/manager.rs:346
6732023-12-15T01:12:02.271ZINFOSledAgent (StorageManager): Received GetLatestResources(Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) }) file = sled-storage/src/manager.rs:346
6742023-12-15T01:12:02.271ZINFOSledAgent (StorageManager): Received GetLatestResources(Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) }) file = sled-storage/src/manager.rs:346
6752023-12-15T01:12:02.271ZINFOSledAgent (StorageManager): Received GetLatestResources(Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) }) file = sled-storage/src/manager.rs:346
6762023-12-15T01:12:02.271ZINFOSledAgent (StorageManager): Received GetLatestResources(Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) }) file = sled-storage/src/manager.rs:346
6772023-12-15T01:12:02.363ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_cockroachdb_7fc3d62c-c7ff-41aa-be0f-d678bc85f00a file = illumos-utils/src/zone.rs:314
6782023-12-15T01:12:02.368ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_cockroachdb_b5d8727f-e19d-4672-a2f8-930bec445061 file = illumos-utils/src/zone.rs:314
6792023-12-15T01:12:02.372ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_cockroachdb_9f6904ab-71f4-4e02-8b40-0deb1fbe77aa file = illumos-utils/src/zone.rs:314
6802023-12-15T01:12:02.377ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_cockroachdb_c80d7f41-e0c5-46c6-b353-df45d1040963 file = illumos-utils/src/zone.rs:314
6812023-12-15T01:12:02.381ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_cockroachdb_00da64ed-baa0-403b-a869-43a61433bbd9 file = illumos-utils/src/zone.rs:314
6822023-12-15T01:12:02.386ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_cockroachdb_7fc3d62c-c7ff-41aa-be0f-d678bc85f00a file = illumos-utils/src/zone.rs:353
6832023-12-15T01:12:02.391ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_cockroachdb_b5d8727f-e19d-4672-a2f8-930bec445061 file = illumos-utils/src/zone.rs:353
6842023-12-15T01:12:02.396ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_cockroachdb_9f6904ab-71f4-4e02-8b40-0deb1fbe77aa file = illumos-utils/src/zone.rs:353
6852023-12-15T01:12:02.400ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_cockroachdb_c80d7f41-e0c5-46c6-b353-df45d1040963 file = illumos-utils/src/zone.rs:353
6862023-12-15T01:12:02.404ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_cockroachdb_00da64ed-baa0-403b-a869-43a61433bbd9 file = illumos-utils/src/zone.rs:353
6872023-12-15T01:12:06.015ZINFOSledAgent (ServiceManager): Profile for oxz_cockroachdb_9f6904ab-71f4-4e02-8b40-0deb1fbe77aa: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/cockroachdb"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value="oxControlService7"/> <propval type="astring" name="gateway" value="fd00:1122:3344:101::1"/> <propval type="astring" name="listen_addr" value="fd00:1122:3344:101::4"/> <propval type="astring" name="listen_port" value="32221"/> <propval type="astring" name="store" value="/data"/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/install"> <property_group type="application" name="install_props"> <property type="net_address" name="nameserver"> <net_address_list> <value_node value="fd00:1122:3344:2::1" /> <value_node value="fd00:1122:3344:1::1" /> <value_node value="fd00:1122:3344:3::1" /> </net_address_list> </property> </property_group> <instance enabled="true" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
6882023-12-15T01:12:06.015ZINFOSledAgent (ServiceManager): Zone booting file = illumos-utils/src/running_zone.rs:512 zone = oxz_cockroachdb_9f6904ab-71f4-4e02-8b40-0deb1fbe77aa
6892023-12-15T01:12:06.093ZINFOSledAgent (ServiceManager): Profile for oxz_cockroachdb_b5d8727f-e19d-4672-a2f8-930bec445061: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/cockroachdb"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value="oxControlService6"/> <propval type="astring" name="gateway" value="fd00:1122:3344:101::1"/> <propval type="astring" name="listen_addr" value="fd00:1122:3344:101::3"/> <propval type="astring" name="listen_port" value="32221"/> <propval type="astring" name="store" value="/data"/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/install"> <property_group type="application" name="install_props"> <property type="net_address" name="nameserver"> <net_address_list> <value_node value="fd00:1122:3344:2::1" /> <value_node value="fd00:1122:3344:1::1" /> <value_node value="fd00:1122:3344:3::1" /> </net_address_list> </property> </property_group> <instance enabled="true" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
6902023-12-15T01:12:06.093ZINFOSledAgent (ServiceManager): Zone booting file = illumos-utils/src/running_zone.rs:512 zone = oxz_cockroachdb_b5d8727f-e19d-4672-a2f8-930bec445061
6912023-12-15T01:12:06.114ZINFOSledAgent (ServiceManager): Profile for oxz_cockroachdb_c80d7f41-e0c5-46c6-b353-df45d1040963: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/cockroachdb"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value="oxControlService8"/> <propval type="astring" name="gateway" value="fd00:1122:3344:101::1"/> <propval type="astring" name="listen_addr" value="fd00:1122:3344:101::5"/> <propval type="astring" name="listen_port" value="32221"/> <propval type="astring" name="store" value="/data"/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/install"> <property_group type="application" name="install_props"> <property type="net_address" name="nameserver"> <net_address_list> <value_node value="fd00:1122:3344:2::1" /> <value_node value="fd00:1122:3344:1::1" /> <value_node value="fd00:1122:3344:3::1" /> </net_address_list> </property> </property_group> <instance enabled="true" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
6922023-12-15T01:12:06.115ZINFOSledAgent (ServiceManager): Zone booting file = illumos-utils/src/running_zone.rs:512 zone = oxz_cockroachdb_c80d7f41-e0c5-46c6-b353-df45d1040963
6932023-12-15T01:12:06.171ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_9f6904ab-71f4-4e02-8b40-0deb1fbe77aa") failed: Property not found. retry in 27.249575ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_9f6904ab-71f4-4e02-8b40-0deb1fbe77aa
6942023-12-15T01:12:06.208ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_9f6904ab-71f4-4e02-8b40-0deb1fbe77aa") failed: Property not found. retry in 89.687298ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_9f6904ab-71f4-4e02-8b40-0deb1fbe77aa
6952023-12-15T01:12:06.256ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_b5d8727f-e19d-4672-a2f8-930bec445061") failed: Property not found. retry in 30.632652ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_b5d8727f-e19d-4672-a2f8-930bec445061
6962023-12-15T01:12:06.301ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_c80d7f41-e0c5-46c6-b353-df45d1040963") failed: Property not found. retry in 39.521016ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_c80d7f41-e0c5-46c6-b353-df45d1040963
6972023-12-15T01:12:06.343ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_b5d8727f-e19d-4672-a2f8-930bec445061") failed: Property not found. retry in 128.028303ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_b5d8727f-e19d-4672-a2f8-930bec445061
6982023-12-15T01:12:06.478ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_9f6904ab-71f4-4e02-8b40-0deb1fbe77aa") failed: Property not found. retry in 173.207113ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_9f6904ab-71f4-4e02-8b40-0deb1fbe77aa
6992023-12-15T01:12:06.522ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_b5d8727f-e19d-4672-a2f8-930bec445061") failed: Property not found. retry in 283.245986ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_b5d8727f-e19d-4672-a2f8-930bec445061
7002023-12-15T01:12:07.114ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_c80d7f41-e0c5-46c6-b353-df45d1040963") failed: Property not found. retry in 118.937076ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_c80d7f41-e0c5-46c6-b353-df45d1040963
7012023-12-15T01:12:07.152ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::605a:15ff:fea0:c066": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
7022023-12-15T01:12:07.203ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_b5d8727f-e19d-4672-a2f8-930bec445061") failed: Property not found. retry in 357.461211ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_b5d8727f-e19d-4672-a2f8-930bec445061
7032023-12-15T01:12:07.292ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_9f6904ab-71f4-4e02-8b40-0deb1fbe77aa") failed: Property not found. retry in 354.853014ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_9f6904ab-71f4-4e02-8b40-0deb1fbe77aa
7042023-12-15T01:12:07.331ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_c80d7f41-e0c5-46c6-b353-df45d1040963") failed: Property not found. retry in 191.855581ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_c80d7f41-e0c5-46c6-b353-df45d1040963
7052023-12-15T01:12:07.610ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_c80d7f41-e0c5-46c6-b353-df45d1040963") failed: Property not found. retry in 436.46207ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_c80d7f41-e0c5-46c6-b353-df45d1040963
7062023-12-15T01:12:07.649ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_b5d8727f-e19d-4672-a2f8-930bec445061") failed: Property not found. retry in 525.037713ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_b5d8727f-e19d-4672-a2f8-930bec445061
7072023-12-15T01:12:07.674ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_9f6904ab-71f4-4e02-8b40-0deb1fbe77aa") failed: Property not found. retry in 920.114062ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_9f6904ab-71f4-4e02-8b40-0deb1fbe77aa
7082023-12-15T01:12:08.058ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_c80d7f41-e0c5-46c6-b353-df45d1040963") failed: Property not found. retry in 1.087561671s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_c80d7f41-e0c5-46c6-b353-df45d1040963
7092023-12-15T01:12:08.185ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_b5d8727f-e19d-4672-a2f8-930bec445061") failed: Property not found. retry in 894.369059ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_b5d8727f-e19d-4672-a2f8-930bec445061
7102023-12-15T01:12:08.340ZINFOSledAgent (ServiceManager): Profile for oxz_cockroachdb_7fc3d62c-c7ff-41aa-be0f-d678bc85f00a: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/cockroachdb"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value="oxControlService5"/> <propval type="astring" name="gateway" value="fd00:1122:3344:101::1"/> <propval type="astring" name="listen_addr" value="fd00:1122:3344:101::7"/> <propval type="astring" name="listen_port" value="32221"/> <propval type="astring" name="store" value="/data"/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/install"> <property_group type="application" name="install_props"> <property type="net_address" name="nameserver"> <net_address_list> <value_node value="fd00:1122:3344:2::1" /> <value_node value="fd00:1122:3344:1::1" /> <value_node value="fd00:1122:3344:3::1" /> </net_address_list> </property> </property_group> <instance enabled="true" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
7112023-12-15T01:12:08.341ZINFOSledAgent (ServiceManager): Zone booting file = illumos-utils/src/running_zone.rs:512 zone = oxz_cockroachdb_7fc3d62c-c7ff-41aa-be0f-d678bc85f00a
7122023-12-15T01:12:08.410ZINFOSledAgent (ServiceManager): Profile for oxz_cockroachdb_00da64ed-baa0-403b-a869-43a61433bbd9: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/cockroachdb"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value="oxControlService9"/> <propval type="astring" name="gateway" value="fd00:1122:3344:101::1"/> <propval type="astring" name="listen_addr" value="fd00:1122:3344:101::6"/> <propval type="astring" name="listen_port" value="32221"/> <propval type="astring" name="store" value="/data"/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/install"> <property_group type="application" name="install_props"> <property type="net_address" name="nameserver"> <net_address_list> <value_node value="fd00:1122:3344:2::1" /> <value_node value="fd00:1122:3344:1::1" /> <value_node value="fd00:1122:3344:3::1" /> </net_address_list> </property> </property_group> <instance enabled="true" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
7132023-12-15T01:12:08.410ZINFOSledAgent (ServiceManager): Zone booting file = illumos-utils/src/running_zone.rs:512 zone = oxz_cockroachdb_00da64ed-baa0-403b-a869-43a61433bbd9
7142023-12-15T01:12:08.492ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_7fc3d62c-c7ff-41aa-be0f-d678bc85f00a") failed: Property not found. retry in 65.968211ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_7fc3d62c-c7ff-41aa-be0f-d678bc85f00a
7152023-12-15T01:12:08.562ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_00da64ed-baa0-403b-a869-43a61433bbd9") failed: Property not found. retry in 47.715988ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_00da64ed-baa0-403b-a869-43a61433bbd9
7162023-12-15T01:12:08.576ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_7fc3d62c-c7ff-41aa-be0f-d678bc85f00a") failed: Property not found. retry in 118.686287ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_7fc3d62c-c7ff-41aa-be0f-d678bc85f00a
7172023-12-15T01:12:08.607ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_9f6904ab-71f4-4e02-8b40-0deb1fbe77aa") failed: Property not found. retry in 989.862481ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_9f6904ab-71f4-4e02-8b40-0deb1fbe77aa
7182023-12-15T01:12:08.620ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_00da64ed-baa0-403b-a869-43a61433bbd9") failed: Property not found. retry in 104.206323ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_00da64ed-baa0-403b-a869-43a61433bbd9
7192023-12-15T01:12:08.710ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_7fc3d62c-c7ff-41aa-be0f-d678bc85f00a") failed: Property not found. retry in 165.340306ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_7fc3d62c-c7ff-41aa-be0f-d678bc85f00a
7202023-12-15T01:12:08.737ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_00da64ed-baa0-403b-a869-43a61433bbd9") failed: Property not found. retry in 149.82734ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_00da64ed-baa0-403b-a869-43a61433bbd9
7212023-12-15T01:12:08.887ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_7fc3d62c-c7ff-41aa-be0f-d678bc85f00a") failed: Property not found. retry in 384.828919ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_7fc3d62c-c7ff-41aa-be0f-d678bc85f00a
7222023-12-15T01:12:08.900ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_00da64ed-baa0-403b-a869-43a61433bbd9") failed: Property not found. retry in 277.447179ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_00da64ed-baa0-403b-a869-43a61433bbd9
7232023-12-15T01:12:09.089ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_b5d8727f-e19d-4672-a2f8-930bec445061") failed: Property not found. retry in 674.677472ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_b5d8727f-e19d-4672-a2f8-930bec445061
7242023-12-15T01:12:09.155ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_c80d7f41-e0c5-46c6-b353-df45d1040963") failed: Property not found. retry in 608.541095ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_c80d7f41-e0c5-46c6-b353-df45d1040963
7252023-12-15T01:12:09.187ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_00da64ed-baa0-403b-a869-43a61433bbd9") failed: Property not found. retry in 671.179307ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_00da64ed-baa0-403b-a869-43a61433bbd9
7262023-12-15T01:12:09.282ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_7fc3d62c-c7ff-41aa-be0f-d678bc85f00a") failed: Property not found. retry in 1.160182702s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_7fc3d62c-c7ff-41aa-be0f-d678bc85f00a
7272023-12-15T01:12:09.608ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_9f6904ab-71f4-4e02-8b40-0deb1fbe77aa") failed: Property not found. retry in 910.723588ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_9f6904ab-71f4-4e02-8b40-0deb1fbe77aa
7282023-12-15T01:12:09.776ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_b5d8727f-e19d-4672-a2f8-930bec445061") failed: Property not found. retry in 791.943872ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_b5d8727f-e19d-4672-a2f8-930bec445061
7292023-12-15T01:12:09.787ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_c80d7f41-e0c5-46c6-b353-df45d1040963") failed: Property not found. retry in 1.035681654s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_c80d7f41-e0c5-46c6-b353-df45d1040963
7302023-12-15T01:12:09.867ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_00da64ed-baa0-403b-a869-43a61433bbd9") failed: Property not found. retry in 1.409524928s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_00da64ed-baa0-403b-a869-43a61433bbd9
7312023-12-15T01:12:10.453ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_7fc3d62c-c7ff-41aa-be0f-d678bc85f00a") failed: Property not found. retry in 1.134078895s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_7fc3d62c-c7ff-41aa-be0f-d678bc85f00a
7322023-12-15T01:12:10.531ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_9f6904ab-71f4-4e02-8b40-0deb1fbe77aa") failed: Property not found. retry in 938.050007ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_9f6904ab-71f4-4e02-8b40-0deb1fbe77aa
7332023-12-15T01:12:10.579ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_b5d8727f-e19d-4672-a2f8-930bec445061") failed: Property not found. retry in 928.658186ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_b5d8727f-e19d-4672-a2f8-930bec445061
7342023-12-15T01:12:10.832ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_c80d7f41-e0c5-46c6-b353-df45d1040963") failed: Property not found. retry in 1.256222021s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_c80d7f41-e0c5-46c6-b353-df45d1040963
7352023-12-15T01:12:11.287ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_00da64ed-baa0-403b-a869-43a61433bbd9") failed: Property not found. retry in 853.073183ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_00da64ed-baa0-403b-a869-43a61433bbd9
7362023-12-15T01:12:11.478ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_9f6904ab-71f4-4e02-8b40-0deb1fbe77aa") failed: Property not found. retry in 673.273186ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_9f6904ab-71f4-4e02-8b40-0deb1fbe77aa
7372023-12-15T01:12:11.516ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_b5d8727f-e19d-4672-a2f8-930bec445061") failed: Property not found. retry in 601.70604ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_b5d8727f-e19d-4672-a2f8-930bec445061
7382023-12-15T01:12:11.596ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_7fc3d62c-c7ff-41aa-be0f-d678bc85f00a") failed: Property not found. retry in 1.390284651s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_7fc3d62c-c7ff-41aa-be0f-d678bc85f00a
7392023-12-15T01:12:12.099ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_c80d7f41-e0c5-46c6-b353-df45d1040963") failed: Property not found. retry in 957.387501ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_c80d7f41-e0c5-46c6-b353-df45d1040963
7402023-12-15T01:12:12.127ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_b5d8727f-e19d-4672-a2f8-930bec445061") failed: Property not found. retry in 1.350010925s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_b5d8727f-e19d-4672-a2f8-930bec445061
7412023-12-15T01:12:12.149ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_00da64ed-baa0-403b-a869-43a61433bbd9") failed: Property not found. retry in 1.332685325s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_00da64ed-baa0-403b-a869-43a61433bbd9
7422023-12-15T01:12:12.159ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_9f6904ab-71f4-4e02-8b40-0deb1fbe77aa") failed: Property not found. retry in 619.748ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_9f6904ab-71f4-4e02-8b40-0deb1fbe77aa
7432023-12-15T01:12:12.159ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::605a:15ff:fea0:c066": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
7442023-12-15T01:12:12.789ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_9f6904ab-71f4-4e02-8b40-0deb1fbe77aa") failed: Property not found. retry in 621.521972ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_9f6904ab-71f4-4e02-8b40-0deb1fbe77aa
7452023-12-15T01:12:12.998ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_7fc3d62c-c7ff-41aa-be0f-d678bc85f00a") failed: Property not found. retry in 1.30847588s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_7fc3d62c-c7ff-41aa-be0f-d678bc85f00a
7462023-12-15T01:12:13.068ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_c80d7f41-e0c5-46c6-b353-df45d1040963") failed: Property not found. retry in 501.302338ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_c80d7f41-e0c5-46c6-b353-df45d1040963
7472023-12-15T01:12:13.421ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_9f6904ab-71f4-4e02-8b40-0deb1fbe77aa") failed: Property not found. retry in 1.019392769s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_9f6904ab-71f4-4e02-8b40-0deb1fbe77aa
7482023-12-15T01:12:13.486ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_b5d8727f-e19d-4672-a2f8-930bec445061") failed: Property not found. retry in 681.385692ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_b5d8727f-e19d-4672-a2f8-930bec445061
7492023-12-15T01:12:13.495ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_00da64ed-baa0-403b-a869-43a61433bbd9") failed: Property not found. retry in 984.353566ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_00da64ed-baa0-403b-a869-43a61433bbd9
7502023-12-15T01:12:13.578ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_c80d7f41-e0c5-46c6-b353-df45d1040963") failed: Property not found. retry in 1.307384992s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_c80d7f41-e0c5-46c6-b353-df45d1040963
7512023-12-15T01:12:14.177ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_b5d8727f-e19d-4672-a2f8-930bec445061") failed: Property not found. retry in 1.393184506s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_b5d8727f-e19d-4672-a2f8-930bec445061
7522023-12-15T01:12:14.316ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_7fc3d62c-c7ff-41aa-be0f-d678bc85f00a") failed: Property not found. retry in 1.337879077s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_7fc3d62c-c7ff-41aa-be0f-d678bc85f00a
7532023-12-15T01:12:14.449ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_9f6904ab-71f4-4e02-8b40-0deb1fbe77aa") failed: Property not found. retry in 1.17940078s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_9f6904ab-71f4-4e02-8b40-0deb1fbe77aa
7542023-12-15T01:12:14.487ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_00da64ed-baa0-403b-a869-43a61433bbd9") failed: Property not found. retry in 1.264405127s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_00da64ed-baa0-403b-a869-43a61433bbd9
7552023-12-15T01:12:14.895ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_c80d7f41-e0c5-46c6-b353-df45d1040963") failed: Property not found. retry in 1.398088654s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_c80d7f41-e0c5-46c6-b353-df45d1040963
7562023-12-15T01:12:15.579ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_b5d8727f-e19d-4672-a2f8-930bec445061") failed: Property not found. retry in 1.269540911s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_b5d8727f-e19d-4672-a2f8-930bec445061
7572023-12-15T01:12:15.638ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_9f6904ab-71f4-4e02-8b40-0deb1fbe77aa") failed: Property not found. retry in 1.312070623s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_9f6904ab-71f4-4e02-8b40-0deb1fbe77aa
7582023-12-15T01:12:15.663ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_7fc3d62c-c7ff-41aa-be0f-d678bc85f00a") failed: Property not found. retry in 881.088115ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_7fc3d62c-c7ff-41aa-be0f-d678bc85f00a
7592023-12-15T01:12:15.761ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_00da64ed-baa0-403b-a869-43a61433bbd9") failed: Property not found. retry in 1.393468201s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_00da64ed-baa0-403b-a869-43a61433bbd9
7602023-12-15T01:12:16.302ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_c80d7f41-e0c5-46c6-b353-df45d1040963") failed: Property not found. retry in 535.334448ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_c80d7f41-e0c5-46c6-b353-df45d1040963
7612023-12-15T01:12:16.554ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_7fc3d62c-c7ff-41aa-be0f-d678bc85f00a") failed: Property not found. retry in 1.275583365s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_7fc3d62c-c7ff-41aa-be0f-d678bc85f00a
7622023-12-15T01:12:16.848ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_c80d7f41-e0c5-46c6-b353-df45d1040963") failed: Property not found. retry in 502.213584ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_c80d7f41-e0c5-46c6-b353-df45d1040963
7632023-12-15T01:12:16.858ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_b5d8727f-e19d-4672-a2f8-930bec445061") failed: Property not found. retry in 1.142767183s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_b5d8727f-e19d-4672-a2f8-930bec445061
7642023-12-15T01:12:16.959ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_9f6904ab-71f4-4e02-8b40-0deb1fbe77aa") failed: Property not found. retry in 773.169282ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_9f6904ab-71f4-4e02-8b40-0deb1fbe77aa
7652023-12-15T01:12:17.165ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_00da64ed-baa0-403b-a869-43a61433bbd9") failed: Property not found. retry in 1.36054291s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_00da64ed-baa0-403b-a869-43a61433bbd9
7662023-12-15T01:12:17.165ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::605a:15ff:fea0:c066": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
7672023-12-15T01:12:17.361ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_c80d7f41-e0c5-46c6-b353-df45d1040963") failed: Property not found. retry in 855.419294ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_c80d7f41-e0c5-46c6-b353-df45d1040963
7682023-12-15T01:12:17.741ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_9f6904ab-71f4-4e02-8b40-0deb1fbe77aa") failed: Property not found. retry in 1.383958751s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_9f6904ab-71f4-4e02-8b40-0deb1fbe77aa
7692023-12-15T01:12:17.837ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_7fc3d62c-c7ff-41aa-be0f-d678bc85f00a") failed: Property not found. retry in 1.15925887s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_7fc3d62c-c7ff-41aa-be0f-d678bc85f00a
7702023-12-15T01:12:18.011ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_b5d8727f-e19d-4672-a2f8-930bec445061") failed: Property not found. retry in 725.649783ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_b5d8727f-e19d-4672-a2f8-930bec445061
7712023-12-15T01:12:18.225ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_c80d7f41-e0c5-46c6-b353-df45d1040963") failed: Property not found. retry in 933.016193ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_c80d7f41-e0c5-46c6-b353-df45d1040963
7722023-12-15T01:12:18.533ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_00da64ed-baa0-403b-a869-43a61433bbd9") failed: Property not found. retry in 766.90952ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_00da64ed-baa0-403b-a869-43a61433bbd9
7732023-12-15T01:12:18.744ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_b5d8727f-e19d-4672-a2f8-930bec445061") failed: Property not found. retry in 532.780368ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_b5d8727f-e19d-4672-a2f8-930bec445061
7742023-12-15T01:12:19.005ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_7fc3d62c-c7ff-41aa-be0f-d678bc85f00a") failed: Property not found. retry in 917.402818ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_7fc3d62c-c7ff-41aa-be0f-d678bc85f00a
7752023-12-15T01:12:19.135ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_9f6904ab-71f4-4e02-8b40-0deb1fbe77aa") failed: Property not found. retry in 1.167678894s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_9f6904ab-71f4-4e02-8b40-0deb1fbe77aa
7762023-12-15T01:12:19.167ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_c80d7f41-e0c5-46c6-b353-df45d1040963") failed: Property not found. retry in 641.96426ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_c80d7f41-e0c5-46c6-b353-df45d1040963
7772023-12-15T01:12:19.286ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_b5d8727f-e19d-4672-a2f8-930bec445061") failed: Property not found. retry in 850.89499ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_b5d8727f-e19d-4672-a2f8-930bec445061
7782023-12-15T01:12:19.309ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_00da64ed-baa0-403b-a869-43a61433bbd9") failed: Property not found. retry in 1.018270232s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_00da64ed-baa0-403b-a869-43a61433bbd9
7792023-12-15T01:12:19.817ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_c80d7f41-e0c5-46c6-b353-df45d1040963") failed: Property not found. retry in 528.904982ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_c80d7f41-e0c5-46c6-b353-df45d1040963
7802023-12-15T01:12:19.930ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_7fc3d62c-c7ff-41aa-be0f-d678bc85f00a") failed: Property not found. retry in 527.755074ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_7fc3d62c-c7ff-41aa-be0f-d678bc85f00a
7812023-12-15T01:12:20.145ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_b5d8727f-e19d-4672-a2f8-930bec445061") failed: Property not found. retry in 838.087399ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_b5d8727f-e19d-4672-a2f8-930bec445061
7822023-12-15T01:12:20.310ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_9f6904ab-71f4-4e02-8b40-0deb1fbe77aa") failed: Property not found. retry in 783.823785ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_9f6904ab-71f4-4e02-8b40-0deb1fbe77aa
7832023-12-15T01:12:20.335ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_00da64ed-baa0-403b-a869-43a61433bbd9") failed: Property not found. retry in 1.125677386s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_00da64ed-baa0-403b-a869-43a61433bbd9
7842023-12-15T01:12:20.355ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_c80d7f41-e0c5-46c6-b353-df45d1040963") failed: Property not found. retry in 534.035091ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_c80d7f41-e0c5-46c6-b353-df45d1040963
7852023-12-15T01:12:20.466ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_7fc3d62c-c7ff-41aa-be0f-d678bc85f00a") failed: Property not found. retry in 965.249484ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_7fc3d62c-c7ff-41aa-be0f-d678bc85f00a
7862023-12-15T01:12:20.899ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_c80d7f41-e0c5-46c6-b353-df45d1040963") failed: Property not found. retry in 585.452947ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_c80d7f41-e0c5-46c6-b353-df45d1040963
7872023-12-15T01:12:20.992ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_b5d8727f-e19d-4672-a2f8-930bec445061") failed: Property not found. retry in 803.604108ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_b5d8727f-e19d-4672-a2f8-930bec445061
7882023-12-15T01:12:21.102ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_9f6904ab-71f4-4e02-8b40-0deb1fbe77aa") failed: Property not found. retry in 1.363473128s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_9f6904ab-71f4-4e02-8b40-0deb1fbe77aa
7892023-12-15T01:12:21.440ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_7fc3d62c-c7ff-41aa-be0f-d678bc85f00a") failed: Property not found. retry in 710.264946ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_7fc3d62c-c7ff-41aa-be0f-d678bc85f00a
7902023-12-15T01:12:21.470ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_00da64ed-baa0-403b-a869-43a61433bbd9") failed: Property not found. retry in 952.94256ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_00da64ed-baa0-403b-a869-43a61433bbd9
7912023-12-15T01:12:21.493ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_c80d7f41-e0c5-46c6-b353-df45d1040963") failed: Property not found. retry in 874.783158ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_c80d7f41-e0c5-46c6-b353-df45d1040963
7922023-12-15T01:12:21.805ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_b5d8727f-e19d-4672-a2f8-930bec445061") failed: Property not found. retry in 786.914425ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_b5d8727f-e19d-4672-a2f8-930bec445061
7932023-12-15T01:12:22.159ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_7fc3d62c-c7ff-41aa-be0f-d678bc85f00a") failed: Property not found. retry in 1.229406599s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_7fc3d62c-c7ff-41aa-be0f-d678bc85f00a
7942023-12-15T01:12:22.167ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::605a:15ff:fea0:c066": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
7952023-12-15T01:12:22.377ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_c80d7f41-e0c5-46c6-b353-df45d1040963") failed: Property not found. retry in 684.236245ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_c80d7f41-e0c5-46c6-b353-df45d1040963
7962023-12-15T01:12:22.432ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_00da64ed-baa0-403b-a869-43a61433bbd9") failed: Property not found. retry in 547.246424ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_00da64ed-baa0-403b-a869-43a61433bbd9
7972023-12-15T01:12:22.474ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_9f6904ab-71f4-4e02-8b40-0deb1fbe77aa") failed: Property not found. retry in 898.704254ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_9f6904ab-71f4-4e02-8b40-0deb1fbe77aa
7982023-12-15T01:12:22.600ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_b5d8727f-e19d-4672-a2f8-930bec445061") failed: Property not found. retry in 1.237597534s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_b5d8727f-e19d-4672-a2f8-930bec445061
7992023-12-15T01:12:22.987ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_00da64ed-baa0-403b-a869-43a61433bbd9") failed: Property not found. retry in 657.738902ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_00da64ed-baa0-403b-a869-43a61433bbd9
8002023-12-15T01:12:23.069ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_c80d7f41-e0c5-46c6-b353-df45d1040963") failed: Property not found. retry in 663.273556ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_c80d7f41-e0c5-46c6-b353-df45d1040963
8012023-12-15T01:12:23.861ZINFOSledAgent (ServiceManager): Writing ledger to /pool/int/a462a7f7-b628-40fe-80ff-4e4189e2d62b/config/.omicron-zones.json.tmp file = common/src/ledger.rs:198
8022023-12-15T01:12:23.861ZINFOSledAgent (ServiceManager): Writing ledger to /pool/int/b462a7f7-b628-40fe-80ff-4e4189e2d62b/config/.omicron-zones.json.tmp file = common/src/ledger.rs:198
8032023-12-15T01:12:23.862ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 21939159 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::1]:55091 req_id = c8dcc70e-b5e7-49a6-b353-c4e05a1b1c74 response_code = 204 uri = /omicron-zones
8042023-12-15T01:12:23.894ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:769 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:35868
8052023-12-15T01:12:23.924ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = Failed to get address: Failed to get address for name oxControlService9/omicron6 in oxz_cockroachdb_00da64ed-baa0-403b-a869-43a61433bbd9: Zone execution error: Command [/usr/sbin/zlogin oxz_cockroachdb_00da64ed-baa0-403b-a869-43a61433bbd9 /usr/sbin/ipadm show-addr -p -o ADDR oxControlService9/omicron6] executed and failed with status: exit status: 1 stdout: \n stderr: ipadm: Address object not found file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:841 latency_us = 30803 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::1]:35868 req_id = 9f21b6d2-8628-4dbc-99bf-da4ebdce7307 response_code = 500 uri = /cockroachdb
8062023-12-15T01:12:23.925ZWARNSledAgent (RSS): Failed to initialize CockroachDB error = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "9f21b6d2-8628-4dbc-99bf-da4ebdce7307", "content-length": "124", "date": "Fri, 15 Dec 2023 01:12:23 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "9f21b6d2-8628-4dbc-99bf-da4ebdce7307" } file = sled-agent/src/rack_setup/service.rs:758 retry_after = 112.105691ms
8072023-12-15T01:12:24.065ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = Failed to get address: Failed to get address for name oxControlService9/omicron6 in oxz_cockroachdb_00da64ed-baa0-403b-a869-43a61433bbd9: Zone execution error: Command [/usr/sbin/zlogin oxz_cockroachdb_00da64ed-baa0-403b-a869-43a61433bbd9 /usr/sbin/ipadm show-addr -p -o ADDR oxControlService9/omicron6] executed and failed with status: exit status: 1 stdout: \n stderr: ipadm: Address object not found file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:841 latency_us = 26688 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::1]:35868 req_id = f1c8c058-34ca-43ab-bb2d-bf2438e51f4e response_code = 500 uri = /cockroachdb
8082023-12-15T01:12:24.065ZWARNSledAgent (RSS): Failed to initialize CockroachDB error = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "f1c8c058-34ca-43ab-bb2d-bf2438e51f4e", "content-length": "124", "date": "Fri, 15 Dec 2023 01:12:23 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "f1c8c058-34ca-43ab-bb2d-bf2438e51f4e" } file = sled-agent/src/rack_setup/service.rs:758 retry_after = 89.901578ms
8092023-12-15T01:12:24.183ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = Failed to get address: Failed to get address for name oxControlService9/omicron6 in oxz_cockroachdb_00da64ed-baa0-403b-a869-43a61433bbd9: Zone execution error: Command [/usr/sbin/zlogin oxz_cockroachdb_00da64ed-baa0-403b-a869-43a61433bbd9 /usr/sbin/ipadm show-addr -p -o ADDR oxControlService9/omicron6] executed and failed with status: exit status: 1 stdout: \n stderr: ipadm: Address object not found file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:841 latency_us = 26552 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::1]:35868 req_id = 7aab8dab-04c7-4eb2-8579-08bdd8478232 response_code = 500 uri = /cockroachdb
8102023-12-15T01:12:24.183ZWARNSledAgent (RSS): Failed to initialize CockroachDB error = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "7aab8dab-04c7-4eb2-8579-08bdd8478232", "content-length": "124", "date": "Fri, 15 Dec 2023 01:12:23 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "7aab8dab-04c7-4eb2-8579-08bdd8478232" } file = sled-agent/src/rack_setup/service.rs:758 retry_after = 152.28695ms
8112023-12-15T01:12:24.368ZINFOSledAgent (ServiceManager): Initializing CRDB Cluster - sending request to [fd00:1122:3344:101::6]:32221 file = sled-agent/src/services.rs:2876
8122023-12-15T01:12:32.298ZINFOSledAgent (ServiceManager): Formatting CRDB file = sled-agent/src/services.rs:2894
8132023-12-15T01:12:34.498ZINFOSledAgent (ServiceManager): Formatting CRDB - Completed file = sled-agent/src/services.rs:2915
8142023-12-15T01:12:34.498ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 10161432 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::1]:35868 req_id = d208ad91-a1e6-48a3-a27b-d487d93b0067 response_code = 204 uri = /cockroachdb
8152023-12-15T01:12:34.498ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:769 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:61967
8162023-12-15T01:12:34.498ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 166 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:61967 req_id = 80a3b39e-376c-4579-b7fd-63c11572b129 response_code = 200 uri = /baseboard
8172023-12-15T01:12:34.500ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::605a:15ff:fea0:c066": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
8182023-12-15T01:12:34.533ZINFOSledAgent (RSS): attempting to set up sled's Omicron zones: OmicronZonesConfig { generation: Generation(5), zones: [OmicronZoneConfig { id: c8d17562-2f06-40f3-9824-1f4cb22a7b12, underlay_address: fd00:1122:3344:1::1, zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 14b4dc87-ab46-49fb-a4b4-d361ae214c03, kind: External } }, http_address: [fd00:1122:3344:1::1]:5353, dns_address: [fd00:1122:3344:1::1]:53, gz_address: fd00:1122:3344:1::2, gz_address_index: 0 } }, OmicronZoneConfig { id: aa7234de-3d19-4031-963b-712389d0323e, underlay_address: fd00:1122:3344:2::1, zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 24b4dc87-ab46-49fb-a4b4-d361ae214c03, kind: External } }, http_address: [fd00:1122:3344:2::1]:5353, dns_address: [fd00:1122:3344:2::1]:53, gz_address: fd00:1122:3344:2::2, gz_address_index: 1 } }, OmicronZoneConfig { id: cc39462d-3d1b-41ac-8720-1bae501935df, underlay_address: fd00:1122:3344:3::1, zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 31bd71cd-4736-4a12-a387-9b74b050396f, kind: External } }, http_address: [fd00:1122:3344:3::1]:5353, dns_address: [fd00:1122:3344:3::1]:53, gz_address: fd00:1122:3344:3::2, gz_address_index: 2 } }, OmicronZoneConfig { id: 6e41b21b-bf42-483b-b26f-9cfe96a37942, underlay_address: fd00:1122:3344:101::1b, zone_type: BoundaryNtp { address: [fd00:1122:3344:101::1b]:123, ntp_servers: ["0.pool.ntp.org"], dns_servers: [1.1.1.1, 9.9.9.9], domain: None, nic: NetworkInterface { id: 06202c6c-62fd-4d5b-8f49-b1be87b00df3, kind: Service { id: 6e41b21b-bf42-483b-b26f-9cfe96a37942 }, name: Name("ntp-6e41b21b-bf42-483b-b26f-9cfe96a37942"), ip: 172.30.3.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 218, 40])), subnet: V4(Ipv4Net(Ipv4Network { addr: 172.30.3.0, prefix: 24 })), vni: Vni(100), primary: true, slot: 0 }, snat_cfg: SourceNatConfig { ip: 10.151.2.169, first_port: 0, last_port: 16383 } } }, OmicronZoneConfig { id: b5d8727f-e19d-4672-a2f8-930bec445061, underlay_address: fd00:1122:3344:101::3, zone_type: CockroachDb { address: [fd00:1122:3344:101::3]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 14b4dc87-ab46-49fb-a4b4-d361ae214c03, kind: External } } } }, OmicronZoneConfig { id: 9f6904ab-71f4-4e02-8b40-0deb1fbe77aa, underlay_address: fd00:1122:3344:101::4, zone_type: CockroachDb { address: [fd00:1122:3344:101::4]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 24b4dc87-ab46-49fb-a4b4-d361ae214c03, kind: External } } } }, OmicronZoneConfig { id: c80d7f41-e0c5-46c6-b353-df45d1040963, underlay_address: fd00:1122:3344:101::5, zone_type: CockroachDb { address: [fd00:1122:3344:101::5]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 31bd71cd-4736-4a12-a387-9b74b050396f, kind: External } } } }, OmicronZoneConfig { id: 00da64ed-baa0-403b-a869-43a61433bbd9, underlay_address: fd00:1122:3344:101::6, zone_type: CockroachDb { address: [fd00:1122:3344:101::6]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 616b26df-e62a-4c68-b506-f4a923d8aaf7, kind: External } } } }, OmicronZoneConfig { id: 7fc3d62c-c7ff-41aa-be0f-d678bc85f00a, underlay_address: fd00:1122:3344:101::7, zone_type: CockroachDb { address: [fd00:1122:3344:101::7]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: cd70d7f6-2354-4bf2-8012-55bf9eaf7930, kind: External } } } }, OmicronZoneConfig { id: 5b1f3c9f-bb97-4778-92ff-b6ab4fa6d3f5, underlay_address: fd00:1122:3344:101::8, zone_type: ExternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 14b4dc87-ab46-49fb-a4b4-d361ae214c03, kind: External } }, http_address: [fd00:1122:3344:101::8]:5353, dns_address: 10.151.2.164:53, nic: NetworkInterface { id: c63661fd-8921-45f4-9ea1-957f627b0035, kind: Service { id: 5b1f3c9f-bb97-4778-92ff-b6ab4fa6d3f5 }, name: Name("external-dns-5b1f3c9f-bb97-4778-92ff-b6ab4fa6d3f5"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 248, 190])), subnet: V4(Ipv4Net(Ipv4Network { addr: 172.30.1.0, prefix: 24 })), vni: Vni(100), primary: true, slot: 0 } } }, OmicronZoneConfig { id: 326e1e2f-55db-4ad4-aa67-d2bb64a872b8, underlay_address: fd00:1122:3344:101::9, zone_type: ExternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 24b4dc87-ab46-49fb-a4b4-d361ae214c03, kind: External } }, http_address: [fd00:1122:3344:101::9]:5353, dns_address: 10.151.2.165:53, nic: NetworkInterface { id: f95722db-ae4d-47db-b31f-4fe3f0644245, kind: Service { id: 326e1e2f-55db-4ad4-aa67-d2bb64a872b8 }, name: Name("external-dns-326e1e2f-55db-4ad4-aa67-d2bb64a872b8"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 155, 183])), subnet: V4(Ipv4Net(Ipv4Network { addr: 172.30.1.0, prefix: 24 })), vni: Vni(100), primary: true, slot: 0 } } }, OmicronZoneConfig { id: 72d8f37c-a72e-4800-9c47-39f6b4e0d24b, underlay_address: fd00:1122:3344:101::a, zone_type: Nexus { internal_address: [fd00:1122:3344:101::a]:12221, external_ip: 10.151.2.166, nic: NetworkInterface { id: ac0cfa1b-6ee0-4c4c-8dd0-6e3f0609fd4a, kind: Service { id: 72d8f37c-a72e-4800-9c47-39f6b4e0d24b }, name: Name("nexus-72d8f37c-a72e-4800-9c47-39f6b4e0d24b"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 160, 115])), subnet: V4(Ipv4Net(Ipv4Network { addr: 172.30.2.0, prefix: 24 })), vni: Vni(100), primary: true, slot: 0 }, external_tls: true, external_dns_servers: [1.1.1.1, 9.9.9.9] } }, OmicronZoneConfig { id: 7198c8dc-786b-4865-a05e-06e4223227c4, underlay_address: fd00:1122:3344:101::b, zone_type: Nexus { internal_address: [fd00:1122:3344:101::b]:12221, external_ip: 10.151.2.167, nic: NetworkInterface { id: 8f13ae46-553d-48d4-9492-805134914028, kind: Service { id: 7198c8dc-786b-4865-a05e-06e4223227c4 }, name: Name("nexus-7198c8dc-786b-4865-a05e-06e4223227c4"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 227, 182])), subnet: V4(Ipv4Net(Ipv4Network { addr: 172.30.2.0, prefix: 24 })), vni: Vni(100), primary: true, slot: 0 }, external_tls: true, external_dns_servers: [1.1.1.1, 9.9.9.9] } }, OmicronZoneConfig { id: 700e7b49-f840-40d5-92e8-83ab758ea8a7, underlay_address: fd00:1122:3344:101::c, zone_type: Nexus { internal_address: [fd00:1122:3344:101::c]:12221, external_ip: 10.151.2.168, nic: NetworkInterface { id: 350830fe-df21-4d4e-9a2e-ab0ea31cf8b3, kind: Service { id: 700e7b49-f840-40d5-92e8-83ab758ea8a7 }, name: Name("nexus-700e7b49-f840-40d5-92e8-83ab758ea8a7"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 230, 76])), subnet: V4(Ipv4Net(Ipv4Network { addr: 172.30.2.0, prefix: 24 })), vni: Vni(100), primary: true, slot: 0 }, external_tls: true, external_dns_servers: [1.1.1.1, 9.9.9.9] } }, OmicronZoneConfig { id: 127efcba-5993-4577-bcb2-f4a98a4520bb, underlay_address: fd00:1122:3344:101::d, zone_type: Oximeter { address: [fd00:1122:3344:101::d]:12223 } }, OmicronZoneConfig { id: 23e76ebf-1083-415c-b1cc-78b70ea0ee28, underlay_address: fd00:1122:3344:101::e, zone_type: Clickhouse { address: [fd00:1122:3344:101::e]:8123, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 14b4dc87-ab46-49fb-a4b4-d361ae214c03, kind: External } } } }, OmicronZoneConfig { id: df48995f-8d48-4496-b8a4-000ccf1c6fe7, underlay_address: fd00:1122:3344:101::f, zone_type: CruciblePantry { address: [fd00:1122:3344:101::f]:17000 } }, OmicronZoneConfig { id: 3fd5ef26-c842-4d29-8079-afd0b75a8502, underlay_address: fd00:1122:3344:101::10, zone_type: CruciblePantry { address: [fd00:1122:3344:101::10]:17000 } }, OmicronZoneConfig { id: cd0082d8-88ea-4123-9209-754395e437e9, underlay_address: fd00:1122:3344:101::11, zone_type: CruciblePantry { address: [fd00:1122:3344:101::11]:17000 } }, OmicronZoneConfig { id: 8b0e0450-abc8-4227-b368-5e4185b52841, underlay_address: fd00:1122:3344:101::12, zone_type: Crucible { address: [fd00:1122:3344:101::12]:32345, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 14b4dc87-ab46-49fb-a4b4-d361ae214c03, kind: External } } } }, OmicronZoneConfig { id: e9d9360e-0097-44f3-aee0-39dd0bc57e9b, underlay_address: fd00:1122:3344:101::13, zone_type: Crucible { address: [fd00:1122:3344:101::13]:32345, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 24b4dc87-ab46-49fb-a4b4-d361ae214c03, kind: External } } } }, OmicronZoneConfig { id: 88b23891-1726-4275-973f-1bfd9c7d86da, underlay_address: fd00:1122:3344:101::14, zone_type: Crucible { address: [fd00:1122:3344:101::14]:32345, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 31bd71cd-4736-4a12-a387-9b74b050396f, kind: External } } } }, OmicronZoneConfig { id: 2217a76b-b58b-4ada-a4c4-e84eca8d38a4, underlay_address: fd00:1122:3344:101::15, zone_type: Crucible { address: [fd00:1122:3344:101::15]:32345, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 616b26df-e62a-4c68-b506-f4a923d8aaf7, kind: External } } } }, OmicronZoneConfig { id: 826ad2ba-d609-43a3-9165-9e544e735622, underlay_address: fd00:1122:3344:101::16, zone_type: Crucible { address: [fd00:1122:3344:101::16]:32345, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: cd70d7f6-2354-4bf2-8012-55bf9eaf7930, kind: External } } } }, OmicronZoneConfig { id: c710b263-6e30-4f78-bb21-7a9e382a9c52, underlay_address: fd00:1122:3344:101::17, zone_type: Crucible { address: [fd00:1122:3344:101::17]:32345, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: ceb4461c-cf56-4719-ad3c-14430bfdfb60, kind: External } } } }, OmicronZoneConfig { id: 17ef7150-35b2-46ab-918a-0b0af0a9b49d, underlay_address: fd00:1122:3344:101::18, zone_type: Crucible { address: [fd00:1122:3344:101::18]:32345, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: d462a7f7-b628-40fe-80ff-4e4189e2d62b, kind: External } } } }, OmicronZoneConfig { id: 04219130-934c-4126-868a-697ce1f64c38, underlay_address: fd00:1122:3344:101::19, zone_type: Crucible { address: [fd00:1122:3344:101::19]:32345, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: e4b4dc87-ab46-49fb-a4b4-d361ae214c03, kind: External } } } }, OmicronZoneConfig { id: 1756a188-a654-4c2f-b5bb-e2f18ad4474e, underlay_address: fd00:1122:3344:101::1a, zone_type: Crucible { address: [fd00:1122:3344:101::1a]:32345, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: f4b4dc87-ab46-49fb-a4b4-d361ae214c03, kind: External } } } }] } file = sled-agent/src/rack_setup/service.rs:299 sled_address = [fd00:1122:3344:101::1]:12345
8192023-12-15T01:12:34.533ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:769 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:41531
8202023-12-15T01:12:34.534ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: c8d17562-2f06-40f3-9824-1f4cb22a7b12, dataset_name: DatasetName { pool_name: ZpoolName { id: 14b4dc87-ab46-49fb-a4b4-d361ae214c03, kind: External }, kind: InternalDns }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } }) file = sled-storage/src/manager.rs:346
8212023-12-15T01:12:34.534ZINFOSledAgent (StorageManager): add_dataset: NewFilesystemRequest { dataset_id: c8d17562-2f06-40f3-9824-1f4cb22a7b12, dataset_name: DatasetName { pool_name: ZpoolName { id: 14b4dc87-ab46-49fb-a4b4-d361ae214c03, kind: External }, kind: InternalDns }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } } file = sled-storage/src/manager.rs:564
8222023-12-15T01:12:34.560ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: aa7234de-3d19-4031-963b-712389d0323e, dataset_name: DatasetName { pool_name: ZpoolName { id: 24b4dc87-ab46-49fb-a4b4-d361ae214c03, kind: External }, kind: InternalDns }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } }) file = sled-storage/src/manager.rs:346
8232023-12-15T01:12:34.560ZINFOSledAgent (StorageManager): add_dataset: NewFilesystemRequest { dataset_id: aa7234de-3d19-4031-963b-712389d0323e, dataset_name: DatasetName { pool_name: ZpoolName { id: 24b4dc87-ab46-49fb-a4b4-d361ae214c03, kind: External }, kind: InternalDns }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } } file = sled-storage/src/manager.rs:564
8242023-12-15T01:12:34.582ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: cc39462d-3d1b-41ac-8720-1bae501935df, dataset_name: DatasetName { pool_name: ZpoolName { id: 31bd71cd-4736-4a12-a387-9b74b050396f, kind: External }, kind: InternalDns }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } }) file = sled-storage/src/manager.rs:346
8252023-12-15T01:12:34.582ZINFOSledAgent (StorageManager): add_dataset: NewFilesystemRequest { dataset_id: cc39462d-3d1b-41ac-8720-1bae501935df, dataset_name: DatasetName { pool_name: ZpoolName { id: 31bd71cd-4736-4a12-a387-9b74b050396f, kind: External }, kind: InternalDns }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } } file = sled-storage/src/manager.rs:564
8262023-12-15T01:12:34.601ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: b5d8727f-e19d-4672-a2f8-930bec445061, dataset_name: DatasetName { pool_name: ZpoolName { id: 14b4dc87-ab46-49fb-a4b4-d361ae214c03, kind: External }, kind: CockroachDb }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } }) file = sled-storage/src/manager.rs:346
8272023-12-15T01:12:34.601ZINFOSledAgent (StorageManager): add_dataset: NewFilesystemRequest { dataset_id: b5d8727f-e19d-4672-a2f8-930bec445061, dataset_name: DatasetName { pool_name: ZpoolName { id: 14b4dc87-ab46-49fb-a4b4-d361ae214c03, kind: External }, kind: CockroachDb }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } } file = sled-storage/src/manager.rs:564
8282023-12-15T01:12:34.619ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 9f6904ab-71f4-4e02-8b40-0deb1fbe77aa, dataset_name: DatasetName { pool_name: ZpoolName { id: 24b4dc87-ab46-49fb-a4b4-d361ae214c03, kind: External }, kind: CockroachDb }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } }) file = sled-storage/src/manager.rs:346
8292023-12-15T01:12:34.619ZINFOSledAgent (StorageManager): add_dataset: NewFilesystemRequest { dataset_id: 9f6904ab-71f4-4e02-8b40-0deb1fbe77aa, dataset_name: DatasetName { pool_name: ZpoolName { id: 24b4dc87-ab46-49fb-a4b4-d361ae214c03, kind: External }, kind: CockroachDb }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } } file = sled-storage/src/manager.rs:564
8302023-12-15T01:12:34.639ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: c80d7f41-e0c5-46c6-b353-df45d1040963, dataset_name: DatasetName { pool_name: ZpoolName { id: 31bd71cd-4736-4a12-a387-9b74b050396f, kind: External }, kind: CockroachDb }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } }) file = sled-storage/src/manager.rs:346
8312023-12-15T01:12:34.639ZINFOSledAgent (StorageManager): add_dataset: NewFilesystemRequest { dataset_id: c80d7f41-e0c5-46c6-b353-df45d1040963, dataset_name: DatasetName { pool_name: ZpoolName { id: 31bd71cd-4736-4a12-a387-9b74b050396f, kind: External }, kind: CockroachDb }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } } file = sled-storage/src/manager.rs:564
8322023-12-15T01:12:34.659ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 00da64ed-baa0-403b-a869-43a61433bbd9, dataset_name: DatasetName { pool_name: ZpoolName { id: 616b26df-e62a-4c68-b506-f4a923d8aaf7, kind: External }, kind: CockroachDb }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } }) file = sled-storage/src/manager.rs:346
8332023-12-15T01:12:34.659ZINFOSledAgent (StorageManager): add_dataset: NewFilesystemRequest { dataset_id: 00da64ed-baa0-403b-a869-43a61433bbd9, dataset_name: DatasetName { pool_name: ZpoolName { id: 616b26df-e62a-4c68-b506-f4a923d8aaf7, kind: External }, kind: CockroachDb }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } } file = sled-storage/src/manager.rs:564
8342023-12-15T01:12:34.679ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 7fc3d62c-c7ff-41aa-be0f-d678bc85f00a, dataset_name: DatasetName { pool_name: ZpoolName { id: cd70d7f6-2354-4bf2-8012-55bf9eaf7930, kind: External }, kind: CockroachDb }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } }) file = sled-storage/src/manager.rs:346
8352023-12-15T01:12:34.679ZINFOSledAgent (StorageManager): add_dataset: NewFilesystemRequest { dataset_id: 7fc3d62c-c7ff-41aa-be0f-d678bc85f00a, dataset_name: DatasetName { pool_name: ZpoolName { id: cd70d7f6-2354-4bf2-8012-55bf9eaf7930, kind: External }, kind: CockroachDb }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } } file = sled-storage/src/manager.rs:564
8362023-12-15T01:12:34.698ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 5b1f3c9f-bb97-4778-92ff-b6ab4fa6d3f5, dataset_name: DatasetName { pool_name: ZpoolName { id: 14b4dc87-ab46-49fb-a4b4-d361ae214c03, kind: External }, kind: ExternalDns }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } }) file = sled-storage/src/manager.rs:346
8372023-12-15T01:12:34.698ZINFOSledAgent (StorageManager): add_dataset: NewFilesystemRequest { dataset_id: 5b1f3c9f-bb97-4778-92ff-b6ab4fa6d3f5, dataset_name: DatasetName { pool_name: ZpoolName { id: 14b4dc87-ab46-49fb-a4b4-d361ae214c03, kind: External }, kind: ExternalDns }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } } file = sled-storage/src/manager.rs:564
8382023-12-15T01:12:34.773ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 326e1e2f-55db-4ad4-aa67-d2bb64a872b8, dataset_name: DatasetName { pool_name: ZpoolName { id: 24b4dc87-ab46-49fb-a4b4-d361ae214c03, kind: External }, kind: ExternalDns }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } }) file = sled-storage/src/manager.rs:346
8392023-12-15T01:12:34.773ZINFOSledAgent (StorageManager): add_dataset: NewFilesystemRequest { dataset_id: 326e1e2f-55db-4ad4-aa67-d2bb64a872b8, dataset_name: DatasetName { pool_name: ZpoolName { id: 24b4dc87-ab46-49fb-a4b4-d361ae214c03, kind: External }, kind: ExternalDns }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } } file = sled-storage/src/manager.rs:564
8402023-12-15T01:12:34.840ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 23e76ebf-1083-415c-b1cc-78b70ea0ee28, dataset_name: DatasetName { pool_name: ZpoolName { id: 14b4dc87-ab46-49fb-a4b4-d361ae214c03, kind: External }, kind: Clickhouse }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } }) file = sled-storage/src/manager.rs:346
8412023-12-15T01:12:34.840ZINFOSledAgent (StorageManager): add_dataset: NewFilesystemRequest { dataset_id: 23e76ebf-1083-415c-b1cc-78b70ea0ee28, dataset_name: DatasetName { pool_name: ZpoolName { id: 14b4dc87-ab46-49fb-a4b4-d361ae214c03, kind: External }, kind: Clickhouse }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } } file = sled-storage/src/manager.rs:564
8422023-12-15T01:12:34.909ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 8b0e0450-abc8-4227-b368-5e4185b52841, dataset_name: DatasetName { pool_name: ZpoolName { id: 14b4dc87-ab46-49fb-a4b4-d361ae214c03, kind: External }, kind: Crucible }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } }) file = sled-storage/src/manager.rs:346
8432023-12-15T01:12:34.909ZINFOSledAgent (StorageManager): add_dataset: NewFilesystemRequest { dataset_id: 8b0e0450-abc8-4227-b368-5e4185b52841, dataset_name: DatasetName { pool_name: ZpoolName { id: 14b4dc87-ab46-49fb-a4b4-d361ae214c03, kind: External }, kind: Crucible }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } } file = sled-storage/src/manager.rs:564
8442023-12-15T01:12:34.970ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: e9d9360e-0097-44f3-aee0-39dd0bc57e9b, dataset_name: DatasetName { pool_name: ZpoolName { id: 24b4dc87-ab46-49fb-a4b4-d361ae214c03, kind: External }, kind: Crucible }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } }) file = sled-storage/src/manager.rs:346
8452023-12-15T01:12:34.970ZINFOSledAgent (StorageManager): add_dataset: NewFilesystemRequest { dataset_id: e9d9360e-0097-44f3-aee0-39dd0bc57e9b, dataset_name: DatasetName { pool_name: ZpoolName { id: 24b4dc87-ab46-49fb-a4b4-d361ae214c03, kind: External }, kind: Crucible }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } } file = sled-storage/src/manager.rs:564
8462023-12-15T01:12:35.034ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 88b23891-1726-4275-973f-1bfd9c7d86da, dataset_name: DatasetName { pool_name: ZpoolName { id: 31bd71cd-4736-4a12-a387-9b74b050396f, kind: External }, kind: Crucible }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } }) file = sled-storage/src/manager.rs:346
8472023-12-15T01:12:35.036ZINFOSledAgent (StorageManager): add_dataset: NewFilesystemRequest { dataset_id: 88b23891-1726-4275-973f-1bfd9c7d86da, dataset_name: DatasetName { pool_name: ZpoolName { id: 31bd71cd-4736-4a12-a387-9b74b050396f, kind: External }, kind: Crucible }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } } file = sled-storage/src/manager.rs:564
8482023-12-15T01:12:35.102ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 2217a76b-b58b-4ada-a4c4-e84eca8d38a4, dataset_name: DatasetName { pool_name: ZpoolName { id: 616b26df-e62a-4c68-b506-f4a923d8aaf7, kind: External }, kind: Crucible }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } }) file = sled-storage/src/manager.rs:346
8492023-12-15T01:12:35.102ZINFOSledAgent (StorageManager): add_dataset: NewFilesystemRequest { dataset_id: 2217a76b-b58b-4ada-a4c4-e84eca8d38a4, dataset_name: DatasetName { pool_name: ZpoolName { id: 616b26df-e62a-4c68-b506-f4a923d8aaf7, kind: External }, kind: Crucible }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } } file = sled-storage/src/manager.rs:564
8502023-12-15T01:12:35.172ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 826ad2ba-d609-43a3-9165-9e544e735622, dataset_name: DatasetName { pool_name: ZpoolName { id: cd70d7f6-2354-4bf2-8012-55bf9eaf7930, kind: External }, kind: Crucible }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } }) file = sled-storage/src/manager.rs:346
8512023-12-15T01:12:35.172ZINFOSledAgent (StorageManager): add_dataset: NewFilesystemRequest { dataset_id: 826ad2ba-d609-43a3-9165-9e544e735622, dataset_name: DatasetName { pool_name: ZpoolName { id: cd70d7f6-2354-4bf2-8012-55bf9eaf7930, kind: External }, kind: Crucible }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } } file = sled-storage/src/manager.rs:564
8522023-12-15T01:12:35.237ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: c710b263-6e30-4f78-bb21-7a9e382a9c52, dataset_name: DatasetName { pool_name: ZpoolName { id: ceb4461c-cf56-4719-ad3c-14430bfdfb60, kind: External }, kind: Crucible }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } }) file = sled-storage/src/manager.rs:346
8532023-12-15T01:12:35.237ZINFOSledAgent (StorageManager): add_dataset: NewFilesystemRequest { dataset_id: c710b263-6e30-4f78-bb21-7a9e382a9c52, dataset_name: DatasetName { pool_name: ZpoolName { id: ceb4461c-cf56-4719-ad3c-14430bfdfb60, kind: External }, kind: Crucible }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } } file = sled-storage/src/manager.rs:564
8542023-12-15T01:12:35.301ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 17ef7150-35b2-46ab-918a-0b0af0a9b49d, dataset_name: DatasetName { pool_name: ZpoolName { id: d462a7f7-b628-40fe-80ff-4e4189e2d62b, kind: External }, kind: Crucible }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } }) file = sled-storage/src/manager.rs:346
8552023-12-15T01:12:35.301ZINFOSledAgent (StorageManager): add_dataset: NewFilesystemRequest { dataset_id: 17ef7150-35b2-46ab-918a-0b0af0a9b49d, dataset_name: DatasetName { pool_name: ZpoolName { id: d462a7f7-b628-40fe-80ff-4e4189e2d62b, kind: External }, kind: Crucible }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } } file = sled-storage/src/manager.rs:564
8562023-12-15T01:12:35.372ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 04219130-934c-4126-868a-697ce1f64c38, dataset_name: DatasetName { pool_name: ZpoolName { id: e4b4dc87-ab46-49fb-a4b4-d361ae214c03, kind: External }, kind: Crucible }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } }) file = sled-storage/src/manager.rs:346
8572023-12-15T01:12:35.372ZINFOSledAgent (StorageManager): add_dataset: NewFilesystemRequest { dataset_id: 04219130-934c-4126-868a-697ce1f64c38, dataset_name: DatasetName { pool_name: ZpoolName { id: e4b4dc87-ab46-49fb-a4b4-d361ae214c03, kind: External }, kind: Crucible }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } } file = sled-storage/src/manager.rs:564
8582023-12-15T01:12:35.428ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 1756a188-a654-4c2f-b5bb-e2f18ad4474e, dataset_name: DatasetName { pool_name: ZpoolName { id: f4b4dc87-ab46-49fb-a4b4-d361ae214c03, kind: External }, kind: Crucible }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } }) file = sled-storage/src/manager.rs:346
8592023-12-15T01:12:35.428ZINFOSledAgent (StorageManager): add_dataset: NewFilesystemRequest { dataset_id: 1756a188-a654-4c2f-b5bb-e2f18ad4474e, dataset_name: DatasetName { pool_name: ZpoolName { id: f4b4dc87-ab46-49fb-a4b4-d361ae214c03, kind: External }, kind: Crucible }, responder: Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) } } file = sled-storage/src/manager.rs:564
8602023-12-15T01:12:35.494ZINFOSledAgent (StorageManager): Received GetLatestResources(Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) }) file = sled-storage/src/manager.rs:346
8612023-12-15T01:12:35.494ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/a462a7f7-b628-40fe-80ff-4e4189e2d62b/config/omicron-zones.json file = common/src/ledger.rs:176
8622023-12-15T01:12:35.494ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/b462a7f7-b628-40fe-80ff-4e4189e2d62b/config/omicron-zones.json file = common/src/ledger.rs:176
8632023-12-15T01:12:35.494ZINFOSledAgent (StorageManager): Received GetLatestResources(Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) }) file = sled-storage/src/manager.rs:346
8642023-12-15T01:12:35.497ZINFOSledAgent (ServiceManager): Resolving switch zone addresses in DNS file = sled-agent/src/bootstrap/early_networking.rs:253
8652023-12-15T01:12:35.499ZINFOSledAgent (ServiceManager): Resolving switch zone addresses in DNS file = sled-agent/src/bootstrap/early_networking.rs:253
8662023-12-15T01:12:35.503ZINFOSledAgent (ServiceManager): Resolving switch zone addresses in DNS file = sled-agent/src/bootstrap/early_networking.rs:253
8672023-12-15T01:12:35.503ZINFOSledAgent (StorageManager): Received GetLatestResources(Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) }) file = sled-storage/src/manager.rs:346
8682023-12-15T01:12:35.503ZINFOSledAgent (StorageManager): Received GetLatestResources(Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) }) file = sled-storage/src/manager.rs:346
8692023-12-15T01:12:35.515ZINFOSledAgent (ServiceManager): Resolving switch zone addresses in DNS file = sled-agent/src/bootstrap/early_networking.rs:253
8702023-12-15T01:12:35.515ZINFOSledAgent (StorageManager): Received GetLatestResources(Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) }) file = sled-storage/src/manager.rs:346
8712023-12-15T01:12:35.515ZINFOSledAgent (StorageManager): Received GetLatestResources(Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) }) file = sled-storage/src/manager.rs:346
8722023-12-15T01:12:35.515ZINFOSledAgent (StorageManager): Received GetLatestResources(Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) }) file = sled-storage/src/manager.rs:346
8732023-12-15T01:12:35.516ZINFOSledAgent (StorageManager): Received GetLatestResources(Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) }) file = sled-storage/src/manager.rs:346
8742023-12-15T01:12:35.516ZINFOSledAgent (StorageManager): Received GetLatestResources(Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) }) file = sled-storage/src/manager.rs:346
8752023-12-15T01:12:35.516ZINFOSledAgent (StorageManager): Received GetLatestResources(Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) }) file = sled-storage/src/manager.rs:346
8762023-12-15T01:12:35.516ZINFOSledAgent (StorageManager): Received GetLatestResources(Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) }) file = sled-storage/src/manager.rs:346
8772023-12-15T01:12:35.516ZINFOSledAgent (StorageManager): Received GetLatestResources(Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) }) file = sled-storage/src/manager.rs:346
8782023-12-15T01:12:35.516ZINFOSledAgent (StorageManager): Received GetLatestResources(Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) }) file = sled-storage/src/manager.rs:346
8792023-12-15T01:12:35.520ZINFOSledAgent (ServiceManager): Resolving switch zone addresses in DNS file = sled-agent/src/bootstrap/early_networking.rs:253
8802023-12-15T01:12:35.520ZINFOSledAgent (StorageManager): Received GetLatestResources(Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) }) file = sled-storage/src/manager.rs:346
8812023-12-15T01:12:35.520ZINFOSledAgent (StorageManager): Received GetLatestResources(Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) }) file = sled-storage/src/manager.rs:346
8822023-12-15T01:12:35.520ZINFOSledAgent (StorageManager): Received GetLatestResources(Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) }) file = sled-storage/src/manager.rs:346
8832023-12-15T01:12:35.797ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_pantry_3fd5ef26-c842-4d29-8079-afd0b75a8502 file = illumos-utils/src/zone.rs:314
8842023-12-15T01:12:35.801ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_pantry_cd0082d8-88ea-4123-9209-754395e437e9 file = illumos-utils/src/zone.rs:314
8852023-12-15T01:12:35.806ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_8b0e0450-abc8-4227-b368-5e4185b52841 file = illumos-utils/src/zone.rs:314
8862023-12-15T01:12:35.810ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_pantry_df48995f-8d48-4496-b8a4-000ccf1c6fe7 file = illumos-utils/src/zone.rs:314
8872023-12-15T01:12:35.815ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_e9d9360e-0097-44f3-aee0-39dd0bc57e9b file = illumos-utils/src/zone.rs:314
8882023-12-15T01:12:35.820ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_88b23891-1726-4275-973f-1bfd9c7d86da file = illumos-utils/src/zone.rs:314
8892023-12-15T01:12:35.825ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_c710b263-6e30-4f78-bb21-7a9e382a9c52 file = illumos-utils/src/zone.rs:314
8902023-12-15T01:12:35.830ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_2217a76b-b58b-4ada-a4c4-e84eca8d38a4 file = illumos-utils/src/zone.rs:314
8912023-12-15T01:12:35.835ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_17ef7150-35b2-46ab-918a-0b0af0a9b49d file = illumos-utils/src/zone.rs:314
8922023-12-15T01:12:35.840ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_04219130-934c-4126-868a-697ce1f64c38 file = illumos-utils/src/zone.rs:314
8932023-12-15T01:12:35.846ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_pantry_3fd5ef26-c842-4d29-8079-afd0b75a8502 file = illumos-utils/src/zone.rs:353
8942023-12-15T01:12:35.850ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_pantry_cd0082d8-88ea-4123-9209-754395e437e9 file = illumos-utils/src/zone.rs:353
8952023-12-15T01:12:35.856ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_8b0e0450-abc8-4227-b368-5e4185b52841 file = illumos-utils/src/zone.rs:353
8962023-12-15T01:12:35.860ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_pantry_df48995f-8d48-4496-b8a4-000ccf1c6fe7 file = illumos-utils/src/zone.rs:353
8972023-12-15T01:12:35.865ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_e9d9360e-0097-44f3-aee0-39dd0bc57e9b file = illumos-utils/src/zone.rs:353
8982023-12-15T01:12:35.870ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_88b23891-1726-4275-973f-1bfd9c7d86da file = illumos-utils/src/zone.rs:353
8992023-12-15T01:12:35.876ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_c710b263-6e30-4f78-bb21-7a9e382a9c52 file = illumos-utils/src/zone.rs:353
9002023-12-15T01:12:35.880ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_2217a76b-b58b-4ada-a4c4-e84eca8d38a4 file = illumos-utils/src/zone.rs:353
9012023-12-15T01:12:35.889ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_17ef7150-35b2-46ab-918a-0b0af0a9b49d file = illumos-utils/src/zone.rs:353
9022023-12-15T01:12:35.895ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_04219130-934c-4126-868a-697ce1f64c38 file = illumos-utils/src/zone.rs:353
9032023-12-15T01:12:35.902ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_clickhouse_23e76ebf-1083-415c-b1cc-78b70ea0ee28 file = illumos-utils/src/zone.rs:314
9042023-12-15T01:12:35.909ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_1756a188-a654-4c2f-b5bb-e2f18ad4474e file = illumos-utils/src/zone.rs:314
9052023-12-15T01:12:35.914ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_826ad2ba-d609-43a3-9165-9e544e735622 file = illumos-utils/src/zone.rs:314
9062023-12-15T01:12:35.920ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_oximeter_127efcba-5993-4577-bcb2-f4a98a4520bb file = illumos-utils/src/zone.rs:314
9072023-12-15T01:12:35.928ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_clickhouse_23e76ebf-1083-415c-b1cc-78b70ea0ee28 file = illumos-utils/src/zone.rs:353
9082023-12-15T01:12:35.974ZINFOSledAgent (ServiceManager): Querying MGS to determine switch location addr = fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:270
9092023-12-15T01:12:35.975ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_1756a188-a654-4c2f-b5bb-e2f18ad4474e file = illumos-utils/src/zone.rs:353
9102023-12-15T01:12:35.981ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_826ad2ba-d609-43a3-9165-9e544e735622 file = illumos-utils/src/zone.rs:353
9112023-12-15T01:12:35.988ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_oximeter_127efcba-5993-4577-bcb2-f4a98a4520bb file = illumos-utils/src/zone.rs:353
9122023-12-15T01:12:36.034ZINFOSledAgent (ServiceManager): Querying MGS to determine switch location addr = fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:270
9132023-12-15T01:12:36.082ZINFOSledAgent (ServiceManager): Querying MGS to determine switch location addr = fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:270
9142023-12-15T01:12:36.126ZINFOSledAgent (ServiceManager): Querying MGS to determine switch location addr = fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:270
9152023-12-15T01:12:36.161ZINFOSledAgent (ServiceManager): Querying MGS to determine switch location addr = fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:270
9162023-12-15T01:12:36.164ZINFOSledAgent (ServiceManager): Found Switch0 at fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:295
9172023-12-15T01:12:36.164ZINFOSledAgent (ServiceManager): Successfully looked up all expected switch zone underlay addresses addrs = {Switch0: fd00:1122:3344:101::2} file = sled-agent/src/bootstrap/early_networking.rs:187
9182023-12-15T01:12:36.223ZINFOSledAgent (PortManager): Created OPTE port file = illumos-utils/src/opte/port_manager.rs:396 port = Port { inner: PortInner { name: "opte1", _ip: 172.30.1.5, mac: MacAddr6([168, 64, 37, 255, 248, 190]), slot: 0, vni: Vni { inner: 100 }, gateway: Gateway { mac: MacAddr6([168, 64, 37, 255, 119, 119]), ip: 172.30.1.1 }, vnic: "vopte1" } }
9192023-12-15T01:12:36.223ZINFOSledAgent (ServiceManager): creating NAT entry for service file = sled-agent/src/services.rs:1221 zone_type = external_dns
9202023-12-15T01:12:36.226ZINFOSledAgent (ServiceManager): Found Switch0 at fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:295
9212023-12-15T01:12:36.226ZINFOSledAgent (ServiceManager): Successfully looked up all expected switch zone underlay addresses addrs = {Switch0: fd00:1122:3344:101::2} file = sled-agent/src/bootstrap/early_networking.rs:187
9222023-12-15T01:12:36.284ZINFOSledAgent (PortManager): Created OPTE port file = illumos-utils/src/opte/port_manager.rs:396 port = Port { inner: PortInner { name: "opte2", _ip: 172.30.2.6, mac: MacAddr6([168, 64, 37, 255, 227, 182]), slot: 0, vni: Vni { inner: 100 }, gateway: Gateway { mac: MacAddr6([168, 64, 37, 255, 119, 119]), ip: 172.30.2.1 }, vnic: "vopte2" } }
9232023-12-15T01:12:36.284ZINFOSledAgent (ServiceManager): creating NAT entry for service file = sled-agent/src/services.rs:1221 zone_type = nexus
9242023-12-15T01:12:36.288ZINFOSledAgent (ServiceManager): Found Switch0 at fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:295
9252023-12-15T01:12:36.288ZINFOSledAgent (ServiceManager): Successfully looked up all expected switch zone underlay addresses addrs = {Switch0: fd00:1122:3344:101::2} file = sled-agent/src/bootstrap/early_networking.rs:187
9262023-12-15T01:12:36.341ZINFOSledAgent (PortManager): Created OPTE port file = illumos-utils/src/opte/port_manager.rs:396 port = Port { inner: PortInner { name: "opte3", _ip: 172.30.2.5, mac: MacAddr6([168, 64, 37, 255, 160, 115]), slot: 0, vni: Vni { inner: 100 }, gateway: Gateway { mac: MacAddr6([168, 64, 37, 255, 119, 119]), ip: 172.30.2.1 }, vnic: "vopte3" } }
9272023-12-15T01:12:36.341ZINFOSledAgent (ServiceManager): creating NAT entry for service file = sled-agent/src/services.rs:1221 zone_type = nexus
9282023-12-15T01:12:36.343ZINFOSledAgent (ServiceManager): Found Switch0 at fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:295
9292023-12-15T01:12:36.343ZINFOSledAgent (ServiceManager): Successfully looked up all expected switch zone underlay addresses addrs = {Switch0: fd00:1122:3344:101::2} file = sled-agent/src/bootstrap/early_networking.rs:187
9302023-12-15T01:12:36.400ZINFOSledAgent (PortManager): Created OPTE port file = illumos-utils/src/opte/port_manager.rs:396 port = Port { inner: PortInner { name: "opte4", _ip: 172.30.2.7, mac: MacAddr6([168, 64, 37, 255, 230, 76]), slot: 0, vni: Vni { inner: 100 }, gateway: Gateway { mac: MacAddr6([168, 64, 37, 255, 119, 119]), ip: 172.30.2.1 }, vnic: "vopte4" } }
9312023-12-15T01:12:36.400ZINFOSledAgent (ServiceManager): creating NAT entry for service file = sled-agent/src/services.rs:1221 zone_type = nexus
9322023-12-15T01:12:36.403ZINFOSledAgent (ServiceManager): Found Switch0 at fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:295
9332023-12-15T01:12:36.403ZINFOSledAgent (ServiceManager): Successfully looked up all expected switch zone underlay addresses addrs = {Switch0: fd00:1122:3344:101::2} file = sled-agent/src/bootstrap/early_networking.rs:187
9342023-12-15T01:12:36.453ZINFOSledAgent (PortManager): Created OPTE port file = illumos-utils/src/opte/port_manager.rs:396 port = Port { inner: PortInner { name: "opte5", _ip: 172.30.1.6, mac: MacAddr6([168, 64, 37, 255, 155, 183]), slot: 0, vni: Vni { inner: 100 }, gateway: Gateway { mac: MacAddr6([168, 64, 37, 255, 119, 119]), ip: 172.30.1.1 }, vnic: "vopte5" } }
9352023-12-15T01:12:36.453ZINFOSledAgent (ServiceManager): creating NAT entry for service file = sled-agent/src/services.rs:1221 zone_type = external_dns
9362023-12-15T01:12:36.454ZINFOSledAgent (ServiceManager): no nat entry found for: V4( Ipv4Network { addr: 10.151.2.164, prefix: 32, }, ) file = clients/dpd-client/src/lib.rs:98
9372023-12-15T01:12:36.454ZINFOSledAgent (ServiceManager): creating nat entry for: V4( Ipv4Network { addr: 10.151.2.164, prefix: 32, }, ) file = clients/dpd-client/src/lib.rs:105
9382023-12-15T01:12:36.455ZINFOSledAgent (ServiceManager): creation of nat entry successful for: V4( Ipv4Network { addr: 10.151.2.164, prefix: 32, }, ) file = clients/dpd-client/src/lib.rs:133
9392023-12-15T01:12:36.458ZINFOSledAgent (StorageManager): Received GetLatestResources(Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) }) file = sled-storage/src/manager.rs:346
9402023-12-15T01:12:36.485ZINFOSledAgent (ServiceManager): no nat entry found for: V4( Ipv4Network { addr: 10.151.2.167, prefix: 32, }, ) file = clients/dpd-client/src/lib.rs:98
9412023-12-15T01:12:36.485ZINFOSledAgent (ServiceManager): creating nat entry for: V4( Ipv4Network { addr: 10.151.2.167, prefix: 32, }, ) file = clients/dpd-client/src/lib.rs:105
9422023-12-15T01:12:36.485ZINFOSledAgent (ServiceManager): no nat entry found for: V4( Ipv4Network { addr: 10.151.2.166, prefix: 32, }, ) file = clients/dpd-client/src/lib.rs:98
9432023-12-15T01:12:36.485ZINFOSledAgent (ServiceManager): creating nat entry for: V4( Ipv4Network { addr: 10.151.2.166, prefix: 32, }, ) file = clients/dpd-client/src/lib.rs:105
9442023-12-15T01:12:36.485ZINFOSledAgent (ServiceManager): no nat entry found for: V4( Ipv4Network { addr: 10.151.2.165, prefix: 32, }, ) file = clients/dpd-client/src/lib.rs:98
9452023-12-15T01:12:36.485ZINFOSledAgent (ServiceManager): creating nat entry for: V4( Ipv4Network { addr: 10.151.2.165, prefix: 32, }, ) file = clients/dpd-client/src/lib.rs:105
9462023-12-15T01:12:36.486ZINFOSledAgent (ServiceManager): no nat entry found for: V4( Ipv4Network { addr: 10.151.2.168, prefix: 32, }, ) file = clients/dpd-client/src/lib.rs:98
9472023-12-15T01:12:36.486ZINFOSledAgent (ServiceManager): creating nat entry for: V4( Ipv4Network { addr: 10.151.2.168, prefix: 32, }, ) file = clients/dpd-client/src/lib.rs:105
9482023-12-15T01:12:36.486ZINFOSledAgent (ServiceManager): creation of nat entry successful for: V4( Ipv4Network { addr: 10.151.2.167, prefix: 32, }, ) file = clients/dpd-client/src/lib.rs:133
9492023-12-15T01:12:36.489ZINFOSledAgent (StorageManager): Received GetLatestResources(Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) }) file = sled-storage/src/manager.rs:346
9502023-12-15T01:12:36.514ZINFOSledAgent (ServiceManager): creation of nat entry successful for: V4( Ipv4Network { addr: 10.151.2.166, prefix: 32, }, ) file = clients/dpd-client/src/lib.rs:133
9512023-12-15T01:12:36.517ZINFOSledAgent (StorageManager): Received GetLatestResources(Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) }) file = sled-storage/src/manager.rs:346
9522023-12-15T01:12:36.517ZINFOSledAgent (ServiceManager): creation of nat entry successful for: V4( Ipv4Network { addr: 10.151.2.165, prefix: 32, }, ) file = clients/dpd-client/src/lib.rs:133
9532023-12-15T01:12:36.519ZINFOSledAgent (StorageManager): Received GetLatestResources(Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) }) file = sled-storage/src/manager.rs:346
9542023-12-15T01:12:36.519ZINFOSledAgent (ServiceManager): creation of nat entry successful for: V4( Ipv4Network { addr: 10.151.2.168, prefix: 32, }, ) file = clients/dpd-client/src/lib.rs:133
9552023-12-15T01:12:36.522ZINFOSledAgent (StorageManager): Received GetLatestResources(Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) }) file = sled-storage/src/manager.rs:346
9562023-12-15T01:12:36.522ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_external_dns_5b1f3c9f-bb97-4778-92ff-b6ab4fa6d3f5 file = illumos-utils/src/zone.rs:314
9572023-12-15T01:12:36.608ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_external_dns_5b1f3c9f-bb97-4778-92ff-b6ab4fa6d3f5 file = illumos-utils/src/zone.rs:353
9582023-12-15T01:12:36.614ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_nexus_7198c8dc-786b-4865-a05e-06e4223227c4 file = illumos-utils/src/zone.rs:314
9592023-12-15T01:12:36.619ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_nexus_72d8f37c-a72e-4800-9c47-39f6b4e0d24b file = illumos-utils/src/zone.rs:314
9602023-12-15T01:12:36.626ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_external_dns_326e1e2f-55db-4ad4-aa67-d2bb64a872b8 file = illumos-utils/src/zone.rs:314
9612023-12-15T01:12:36.632ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_nexus_700e7b49-f840-40d5-92e8-83ab758ea8a7 file = illumos-utils/src/zone.rs:314
9622023-12-15T01:12:36.642ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_nexus_7198c8dc-786b-4865-a05e-06e4223227c4 file = illumos-utils/src/zone.rs:353
9632023-12-15T01:12:36.649ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_nexus_72d8f37c-a72e-4800-9c47-39f6b4e0d24b file = illumos-utils/src/zone.rs:353
9642023-12-15T01:12:36.656ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_external_dns_326e1e2f-55db-4ad4-aa67-d2bb64a872b8 file = illumos-utils/src/zone.rs:353
9652023-12-15T01:12:36.662ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_nexus_700e7b49-f840-40d5-92e8-83ab758ea8a7 file = illumos-utils/src/zone.rs:353
9662023-12-15T01:12:39.373ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_pantry_df48995f-8d48-4496-b8a4-000ccf1c6fe7: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/crucible/pantry"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value="oxControlService13"/> <propval type="astring" name="gateway" value="fd00:1122:3344:101::1"/> <propval type="astring" name="listen_addr" value="fd00:1122:3344:101::f"/> <propval type="astring" name="listen_port" value="17000"/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
9672023-12-15T01:12:39.373ZINFOSledAgent (ServiceManager): Zone booting file = illumos-utils/src/running_zone.rs:512 zone = oxz_crucible_pantry_df48995f-8d48-4496-b8a4-000ccf1c6fe7
9682023-12-15T01:12:39.401ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_pantry_3fd5ef26-c842-4d29-8079-afd0b75a8502: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/crucible/pantry"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value="oxControlService10"/> <propval type="astring" name="gateway" value="fd00:1122:3344:101::1"/> <propval type="astring" name="listen_addr" value="fd00:1122:3344:101::10"/> <propval type="astring" name="listen_port" value="17000"/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
9692023-12-15T01:12:39.401ZINFOSledAgent (ServiceManager): Zone booting file = illumos-utils/src/running_zone.rs:512 zone = oxz_crucible_pantry_3fd5ef26-c842-4d29-8079-afd0b75a8502
9702023-12-15T01:12:39.507ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::605a:15ff:fea0:c066": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
9712023-12-15T01:12:39.668ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_c710b263-6e30-4f78-bb21-7a9e382a9c52: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/crucible/agent"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value="oxControlService16"/> <propval type="astring" name="gateway" value="fd00:1122:3344:101::1"/> <propval type="astring" name="dataset" value="oxp_ceb4461c-cf56-4719-ad3c-14430bfdfb60/crucible"/> <propval type="astring" name="listen_addr" value="fd00:1122:3344:101::17"/> <propval type="astring" name="listen_port" value="32345"/> <propval type="astring" name="uuid" value="84d62b3e-e7f4-4444-a4c1-0e99fa03de8b"/> <propval type="astring" name="store" value="/data"/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
9722023-12-15T01:12:39.668ZINFOSledAgent (ServiceManager): Zone booting file = illumos-utils/src/running_zone.rs:512 zone = oxz_crucible_c710b263-6e30-4f78-bb21-7a9e382a9c52
9732023-12-15T01:12:39.692ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_04219130-934c-4126-868a-697ce1f64c38: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/crucible/agent"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value="oxControlService19"/> <propval type="astring" name="gateway" value="fd00:1122:3344:101::1"/> <propval type="astring" name="dataset" value="oxp_e4b4dc87-ab46-49fb-a4b4-d361ae214c03/crucible"/> <propval type="astring" name="listen_addr" value="fd00:1122:3344:101::19"/> <propval type="astring" name="listen_port" value="32345"/> <propval type="astring" name="uuid" value="632503c8-7748-4c72-836c-bd97c4c0e81b"/> <propval type="astring" name="store" value="/data"/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
9742023-12-15T01:12:39.692ZINFOSledAgent (ServiceManager): Zone booting file = illumos-utils/src/running_zone.rs:512 zone = oxz_crucible_04219130-934c-4126-868a-697ce1f64c38
9752023-12-15T01:12:39.809ZINFOSledAgent (ServiceManager): Zone booting file = illumos-utils/src/running_zone.rs:512 zone = oxz_oximeter_127efcba-5993-4577-bcb2-f4a98a4520bb
9762023-12-15T01:12:39.940ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_2217a76b-b58b-4ada-a4c4-e84eca8d38a4: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/crucible/agent"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value="oxControlService17"/> <propval type="astring" name="gateway" value="fd00:1122:3344:101::1"/> <propval type="astring" name="dataset" value="oxp_616b26df-e62a-4c68-b506-f4a923d8aaf7/crucible"/> <propval type="astring" name="listen_addr" value="fd00:1122:3344:101::15"/> <propval type="astring" name="listen_port" value="32345"/> <propval type="astring" name="uuid" value="d70cbbe9-2000-48f8-bd4a-f5741899af0c"/> <propval type="astring" name="store" value="/data"/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
9772023-12-15T01:12:39.941ZINFOSledAgent (ServiceManager): Zone booting file = illumos-utils/src/running_zone.rs:512 zone = oxz_crucible_2217a76b-b58b-4ada-a4c4-e84eca8d38a4
9782023-12-15T01:12:40.192ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_df48995f-8d48-4496-b8a4-000ccf1c6fe7") failed: Property not found. retry in 25.631967ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_df48995f-8d48-4496-b8a4-000ccf1c6fe7
9792023-12-15T01:12:40.217ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_127efcba-5993-4577-bcb2-f4a98a4520bb") failed: Property not found. retry in 27.94948ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_127efcba-5993-4577-bcb2-f4a98a4520bb
9802023-12-15T01:12:40.237ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_c710b263-6e30-4f78-bb21-7a9e382a9c52") failed: Property not found. retry in 48.564053ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_c710b263-6e30-4f78-bb21-7a9e382a9c52
9812023-12-15T01:12:40.258ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_04219130-934c-4126-868a-697ce1f64c38") failed: Property not found. retry in 41.542595ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_04219130-934c-4126-868a-697ce1f64c38
9822023-12-15T01:12:40.281ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_df48995f-8d48-4496-b8a4-000ccf1c6fe7") failed: Property not found. retry in 132.477667ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_df48995f-8d48-4496-b8a4-000ccf1c6fe7
9832023-12-15T01:12:40.295ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_2217a76b-b58b-4ada-a4c4-e84eca8d38a4") failed: Property not found. retry in 30.190377ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_2217a76b-b58b-4ada-a4c4-e84eca8d38a4
9842023-12-15T01:12:40.363ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_127efcba-5993-4577-bcb2-f4a98a4520bb") failed: Property not found. retry in 126.34771ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_127efcba-5993-4577-bcb2-f4a98a4520bb
9852023-12-15T01:12:40.403ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_c710b263-6e30-4f78-bb21-7a9e382a9c52") failed: Property not found. retry in 100.881245ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_c710b263-6e30-4f78-bb21-7a9e382a9c52
9862023-12-15T01:12:40.437ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_2217a76b-b58b-4ada-a4c4-e84eca8d38a4") failed: Property not found. retry in 89.776989ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_2217a76b-b58b-4ada-a4c4-e84eca8d38a4
9872023-12-15T01:12:40.453ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_04219130-934c-4126-868a-697ce1f64c38") failed: Property not found. retry in 115.037325ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_04219130-934c-4126-868a-697ce1f64c38
9882023-12-15T01:12:40.485ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_df48995f-8d48-4496-b8a4-000ccf1c6fe7") failed: Property not found. retry in 205.300867ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_df48995f-8d48-4496-b8a4-000ccf1c6fe7
9892023-12-15T01:12:40.538ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_127efcba-5993-4577-bcb2-f4a98a4520bb") failed: Property not found. retry in 188.961982ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_127efcba-5993-4577-bcb2-f4a98a4520bb
9902023-12-15T01:12:40.560ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_c710b263-6e30-4f78-bb21-7a9e382a9c52") failed: Property not found. retry in 212.088581ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_c710b263-6e30-4f78-bb21-7a9e382a9c52
9912023-12-15T01:12:40.577ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_2217a76b-b58b-4ada-a4c4-e84eca8d38a4") failed: Property not found. retry in 214.835783ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_2217a76b-b58b-4ada-a4c4-e84eca8d38a4
9922023-12-15T01:12:40.603ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_04219130-934c-4126-868a-697ce1f64c38") failed: Property not found. retry in 200.983922ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_04219130-934c-4126-868a-697ce1f64c38
9932023-12-15T01:12:40.777ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_df48995f-8d48-4496-b8a4-000ccf1c6fe7") failed: Property not found. retry in 461.211746ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_df48995f-8d48-4496-b8a4-000ccf1c6fe7
9942023-12-15T01:12:40.800ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_c710b263-6e30-4f78-bb21-7a9e382a9c52") failed: Property not found. retry in 284.746689ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_c710b263-6e30-4f78-bb21-7a9e382a9c52
9952023-12-15T01:12:40.843ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_127efcba-5993-4577-bcb2-f4a98a4520bb") failed: Property not found. retry in 204.227071ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_127efcba-5993-4577-bcb2-f4a98a4520bb
9962023-12-15T01:12:40.869ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_2217a76b-b58b-4ada-a4c4-e84eca8d38a4") failed: Property not found. retry in 427.339199ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_2217a76b-b58b-4ada-a4c4-e84eca8d38a4
9972023-12-15T01:12:40.894ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_04219130-934c-4126-868a-697ce1f64c38") failed: Property not found. retry in 348.764204ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_04219130-934c-4126-868a-697ce1f64c38
9982023-12-15T01:12:41.208ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_127efcba-5993-4577-bcb2-f4a98a4520bb") failed: Property not found. retry in 881.143757ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_127efcba-5993-4577-bcb2-f4a98a4520bb
9992023-12-15T01:12:41.244ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_c710b263-6e30-4f78-bb21-7a9e382a9c52") failed: Property not found. retry in 621.151541ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_c710b263-6e30-4f78-bb21-7a9e382a9c52
10002023-12-15T01:12:41.266ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_df48995f-8d48-4496-b8a4-000ccf1c6fe7") failed: Property not found. retry in 1.043763395s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_df48995f-8d48-4496-b8a4-000ccf1c6fe7
10012023-12-15T01:12:41.285ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_04219130-934c-4126-868a-697ce1f64c38") failed: Property not found. retry in 887.300537ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_04219130-934c-4126-868a-697ce1f64c38
10022023-12-15T01:12:41.318ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_2217a76b-b58b-4ada-a4c4-e84eca8d38a4") failed: Property not found. retry in 458.377641ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_2217a76b-b58b-4ada-a4c4-e84eca8d38a4
10032023-12-15T01:12:41.791ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_2217a76b-b58b-4ada-a4c4-e84eca8d38a4") failed: Property not found. retry in 1.352652331s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_2217a76b-b58b-4ada-a4c4-e84eca8d38a4
10042023-12-15T01:12:41.877ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_c710b263-6e30-4f78-bb21-7a9e382a9c52") failed: Property not found. retry in 1.188124922s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_c710b263-6e30-4f78-bb21-7a9e382a9c52
10052023-12-15T01:12:42.105ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_127efcba-5993-4577-bcb2-f4a98a4520bb") failed: Property not found. retry in 1.199532698s file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_127efcba-5993-4577-bcb2-f4a98a4520bb
10062023-12-15T01:12:42.186ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_04219130-934c-4126-868a-697ce1f64c38") failed: Property not found. retry in 1.100348809s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_04219130-934c-4126-868a-697ce1f64c38
10072023-12-15T01:12:42.324ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_df48995f-8d48-4496-b8a4-000ccf1c6fe7") failed: Property not found. retry in 1.480681327s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_df48995f-8d48-4496-b8a4-000ccf1c6fe7
10082023-12-15T01:12:43.079ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_c710b263-6e30-4f78-bb21-7a9e382a9c52") failed: Property not found. retry in 946.443727ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_c710b263-6e30-4f78-bb21-7a9e382a9c52
10092023-12-15T01:12:43.157ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_2217a76b-b58b-4ada-a4c4-e84eca8d38a4") failed: Property not found. retry in 1.023789855s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_2217a76b-b58b-4ada-a4c4-e84eca8d38a4
10102023-12-15T01:12:43.299ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_04219130-934c-4126-868a-697ce1f64c38") failed: Property not found. retry in 1.066658079s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_04219130-934c-4126-868a-697ce1f64c38
10112023-12-15T01:12:43.316ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_127efcba-5993-4577-bcb2-f4a98a4520bb") failed: Property not found. retry in 1.080019754s file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_127efcba-5993-4577-bcb2-f4a98a4520bb
10122023-12-15T01:12:43.607ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_pantry_cd0082d8-88ea-4123-9209-754395e437e9: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/crucible/pantry"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value="oxControlService11"/> <propval type="astring" name="gateway" value="fd00:1122:3344:101::1"/> <propval type="astring" name="listen_addr" value="fd00:1122:3344:101::11"/> <propval type="astring" name="listen_port" value="17000"/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
10132023-12-15T01:12:43.608ZINFOSledAgent (ServiceManager): Zone booting file = illumos-utils/src/running_zone.rs:512 zone = oxz_crucible_pantry_cd0082d8-88ea-4123-9209-754395e437e9
10142023-12-15T01:12:43.812ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_cd0082d8-88ea-4123-9209-754395e437e9") failed: Property not found. retry in 46.787026ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_cd0082d8-88ea-4123-9209-754395e437e9
10152023-12-15T01:12:43.823ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_df48995f-8d48-4496-b8a4-000ccf1c6fe7") failed: Property not found. retry in 1.441575022s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_df48995f-8d48-4496-b8a4-000ccf1c6fe7
10162023-12-15T01:12:43.871ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_cd0082d8-88ea-4123-9209-754395e437e9") failed: Property not found. retry in 79.663802ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_cd0082d8-88ea-4123-9209-754395e437e9
10172023-12-15T01:12:43.883ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_8b0e0450-abc8-4227-b368-5e4185b52841: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/crucible/agent"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value="oxControlService12"/> <propval type="astring" name="gateway" value="fd00:1122:3344:101::1"/> <propval type="astring" name="dataset" value="oxp_14b4dc87-ab46-49fb-a4b4-d361ae214c03/crucible"/> <propval type="astring" name="listen_addr" value="fd00:1122:3344:101::12"/> <propval type="astring" name="listen_port" value="32345"/> <propval type="astring" name="uuid" value="7b26465c-7471-4b5b-bb54-499186d4ec7c"/> <propval type="astring" name="store" value="/data"/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
10182023-12-15T01:12:43.883ZINFOSledAgent (ServiceManager): Zone booting file = illumos-utils/src/running_zone.rs:512 zone = oxz_crucible_8b0e0450-abc8-4227-b368-5e4185b52841
10192023-12-15T01:12:43.964ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_cd0082d8-88ea-4123-9209-754395e437e9") failed: Property not found. retry in 181.847498ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_cd0082d8-88ea-4123-9209-754395e437e9
10202023-12-15T01:12:44.039ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_c710b263-6e30-4f78-bb21-7a9e382a9c52") failed: Property not found. retry in 549.665466ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_c710b263-6e30-4f78-bb21-7a9e382a9c52
10212023-12-15T01:12:44.040ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_17ef7150-35b2-46ab-918a-0b0af0a9b49d: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/crucible/agent"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value="oxControlService18"/> <propval type="astring" name="gateway" value="fd00:1122:3344:101::1"/> <propval type="astring" name="dataset" value="oxp_d462a7f7-b628-40fe-80ff-4e4189e2d62b/crucible"/> <propval type="astring" name="listen_addr" value="fd00:1122:3344:101::18"/> <propval type="astring" name="listen_port" value="32345"/> <propval type="astring" name="uuid" value="8b334fa0-dbca-450d-98d4-1d41e252f6f4"/> <propval type="astring" name="store" value="/data"/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
10222023-12-15T01:12:44.040ZINFOSledAgent (ServiceManager): Zone booting file = illumos-utils/src/running_zone.rs:512 zone = oxz_crucible_17ef7150-35b2-46ab-918a-0b0af0a9b49d
10232023-12-15T01:12:44.111ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_8b0e0450-abc8-4227-b368-5e4185b52841") failed: Property not found. retry in 68.090332ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_8b0e0450-abc8-4227-b368-5e4185b52841
10242023-12-15T01:12:44.161ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_cd0082d8-88ea-4123-9209-754395e437e9") failed: Property not found. retry in 309.006867ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_cd0082d8-88ea-4123-9209-754395e437e9
10252023-12-15T01:12:44.189ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_8b0e0450-abc8-4227-b368-5e4185b52841") failed: Property not found. retry in 127.217729ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_8b0e0450-abc8-4227-b368-5e4185b52841
10262023-12-15T01:12:44.199ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_2217a76b-b58b-4ada-a4c4-e84eca8d38a4") failed: Property not found. retry in 1.27040047s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_2217a76b-b58b-4ada-a4c4-e84eca8d38a4
10272023-12-15T01:12:44.282ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_17ef7150-35b2-46ab-918a-0b0af0a9b49d") failed: Property not found. retry in 73.585664ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_17ef7150-35b2-46ab-918a-0b0af0a9b49d
10282023-12-15T01:12:44.330ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_8b0e0450-abc8-4227-b368-5e4185b52841") failed: Property not found. retry in 278.601324ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_8b0e0450-abc8-4227-b368-5e4185b52841
10292023-12-15T01:12:44.369ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_17ef7150-35b2-46ab-918a-0b0af0a9b49d") failed: Property not found. retry in 72.888171ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_17ef7150-35b2-46ab-918a-0b0af0a9b49d
10302023-12-15T01:12:44.382ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_04219130-934c-4126-868a-697ce1f64c38") failed: Property not found. retry in 1.132693396s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_04219130-934c-4126-868a-697ce1f64c38
10312023-12-15T01:12:44.408ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_127efcba-5993-4577-bcb2-f4a98a4520bb") failed: Property not found. retry in 1.055788848s file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_127efcba-5993-4577-bcb2-f4a98a4520bb
10322023-12-15T01:12:44.454ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_17ef7150-35b2-46ab-918a-0b0af0a9b49d") failed: Property not found. retry in 231.810182ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_17ef7150-35b2-46ab-918a-0b0af0a9b49d
10332023-12-15T01:12:44.482ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_cd0082d8-88ea-4123-9209-754395e437e9") failed: Property not found. retry in 1.173418649s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_cd0082d8-88ea-4123-9209-754395e437e9
10342023-12-15T01:12:44.511ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::605a:15ff:fea0:c066": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
10352023-12-15T01:12:44.600ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_c710b263-6e30-4f78-bb21-7a9e382a9c52") failed: Property not found. retry in 1.059194972s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_c710b263-6e30-4f78-bb21-7a9e382a9c52
10362023-12-15T01:12:44.621ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_8b0e0450-abc8-4227-b368-5e4185b52841") failed: Property not found. retry in 363.408303ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_8b0e0450-abc8-4227-b368-5e4185b52841
10372023-12-15T01:12:44.699ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_17ef7150-35b2-46ab-918a-0b0af0a9b49d") failed: Property not found. retry in 342.856889ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_17ef7150-35b2-46ab-918a-0b0af0a9b49d
10382023-12-15T01:12:44.740ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_1756a188-a654-4c2f-b5bb-e2f18ad4474e: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/crucible/agent"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value="oxControlService21"/> <propval type="astring" name="gateway" value="fd00:1122:3344:101::1"/> <propval type="astring" name="dataset" value="oxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crucible"/> <propval type="astring" name="listen_addr" value="fd00:1122:3344:101::1a"/> <propval type="astring" name="listen_port" value="32345"/> <propval type="astring" name="uuid" value="f5a9958f-7e04-499c-a1bb-d4b14e3705ef"/> <propval type="astring" name="store" value="/data"/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
10392023-12-15T01:12:44.740ZINFOSledAgent (ServiceManager): Zone booting file = illumos-utils/src/running_zone.rs:512 zone = oxz_crucible_1756a188-a654-4c2f-b5bb-e2f18ad4474e
10402023-12-15T01:12:44.967ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_1756a188-a654-4c2f-b5bb-e2f18ad4474e") failed: Property not found. retry in 34.529314ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_1756a188-a654-4c2f-b5bb-e2f18ad4474e
10412023-12-15T01:12:44.997ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_8b0e0450-abc8-4227-b368-5e4185b52841") failed: Property not found. retry in 523.361922ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_8b0e0450-abc8-4227-b368-5e4185b52841
10422023-12-15T01:12:45.013ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_1756a188-a654-4c2f-b5bb-e2f18ad4474e") failed: Property not found. retry in 96.239466ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_1756a188-a654-4c2f-b5bb-e2f18ad4474e
10432023-12-15T01:12:45.055ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_17ef7150-35b2-46ab-918a-0b0af0a9b49d") failed: Property not found. retry in 514.769188ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_17ef7150-35b2-46ab-918a-0b0af0a9b49d
10442023-12-15T01:12:45.056ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_88b23891-1726-4275-973f-1bfd9c7d86da: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/crucible/agent"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value="oxControlService15"/> <propval type="astring" name="gateway" value="fd00:1122:3344:101::1"/> <propval type="astring" name="dataset" value="oxp_31bd71cd-4736-4a12-a387-9b74b050396f/crucible"/> <propval type="astring" name="listen_addr" value="fd00:1122:3344:101::14"/> <propval type="astring" name="listen_port" value="32345"/> <propval type="astring" name="uuid" value="d6d74012-5055-423f-a34e-e987b2f69e90"/> <propval type="astring" name="store" value="/data"/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
10452023-12-15T01:12:45.057ZINFOSledAgent (ServiceManager): Zone booting file = illumos-utils/src/running_zone.rs:512 zone = oxz_crucible_88b23891-1726-4275-973f-1bfd9c7d86da
10462023-12-15T01:12:45.082ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_826ad2ba-d609-43a3-9165-9e544e735622: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/crucible/agent"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value="oxControlService22"/> <propval type="astring" name="gateway" value="fd00:1122:3344:101::1"/> <propval type="astring" name="dataset" value="oxp_cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crucible"/> <propval type="astring" name="listen_addr" value="fd00:1122:3344:101::16"/> <propval type="astring" name="listen_port" value="32345"/> <propval type="astring" name="uuid" value="e17790cc-c0de-44d9-adf5-73369ee3cb19"/> <propval type="astring" name="store" value="/data"/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
10472023-12-15T01:12:45.082ZINFOSledAgent (ServiceManager): Zone booting file = illumos-utils/src/running_zone.rs:512 zone = oxz_crucible_826ad2ba-d609-43a3-9165-9e544e735622
10482023-12-15T01:12:45.101ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_e9d9360e-0097-44f3-aee0-39dd0bc57e9b: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/crucible/agent"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value="oxControlService14"/> <propval type="astring" name="gateway" value="fd00:1122:3344:101::1"/> <propval type="astring" name="dataset" value="oxp_24b4dc87-ab46-49fb-a4b4-d361ae214c03/crucible"/> <propval type="astring" name="listen_addr" value="fd00:1122:3344:101::13"/> <propval type="astring" name="listen_port" value="32345"/> <propval type="astring" name="uuid" value="4cb19ecc-66c7-4e44-b6fe-fa1f896efcbc"/> <propval type="astring" name="store" value="/data"/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
10492023-12-15T01:12:45.102ZINFOSledAgent (ServiceManager): Zone booting file = illumos-utils/src/running_zone.rs:512 zone = oxz_crucible_e9d9360e-0097-44f3-aee0-39dd0bc57e9b
10502023-12-15T01:12:45.122ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_1756a188-a654-4c2f-b5bb-e2f18ad4474e") failed: Property not found. retry in 147.10829ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_1756a188-a654-4c2f-b5bb-e2f18ad4474e
10512023-12-15T01:12:45.277ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_df48995f-8d48-4496-b8a4-000ccf1c6fe7") failed: Property not found. retry in 609.641918ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_df48995f-8d48-4496-b8a4-000ccf1c6fe7
10522023-12-15T01:12:45.286ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_88b23891-1726-4275-973f-1bfd9c7d86da") failed: Property not found. retry in 61.27736ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_88b23891-1726-4275-973f-1bfd9c7d86da
10532023-12-15T01:12:45.298ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_1756a188-a654-4c2f-b5bb-e2f18ad4474e") failed: Property not found. retry in 254.689372ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_1756a188-a654-4c2f-b5bb-e2f18ad4474e
10542023-12-15T01:12:45.298ZINFOSledAgent (ServiceManager): Zone booting file = illumos-utils/src/running_zone.rs:512 zone = oxz_external_dns_5b1f3c9f-bb97-4778-92ff-b6ab4fa6d3f5
10552023-12-15T01:12:45.310ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_826ad2ba-d609-43a3-9165-9e544e735622") failed: Property not found. retry in 25.937932ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_826ad2ba-d609-43a3-9165-9e544e735622
10562023-12-15T01:12:45.346ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_826ad2ba-d609-43a3-9165-9e544e735622") failed: Property not found. retry in 102.455798ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_826ad2ba-d609-43a3-9165-9e544e735622
10572023-12-15T01:12:45.358ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_e9d9360e-0097-44f3-aee0-39dd0bc57e9b") failed: Property not found. retry in 51.651673ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_e9d9360e-0097-44f3-aee0-39dd0bc57e9b
10582023-12-15T01:12:45.369ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_88b23891-1726-4275-973f-1bfd9c7d86da") failed: Property not found. retry in 125.431529ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_88b23891-1726-4275-973f-1bfd9c7d86da
10592023-12-15T01:12:45.422ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_e9d9360e-0097-44f3-aee0-39dd0bc57e9b") failed: Property not found. retry in 143.879154ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_e9d9360e-0097-44f3-aee0-39dd0bc57e9b
10602023-12-15T01:12:45.460ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_826ad2ba-d609-43a3-9165-9e544e735622") failed: Property not found. retry in 264.32616ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_826ad2ba-d609-43a3-9165-9e544e735622
10612023-12-15T01:12:45.475ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_127efcba-5993-4577-bcb2-f4a98a4520bb") failed: Property not found. retry in 717.995894ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_127efcba-5993-4577-bcb2-f4a98a4520bb
10622023-12-15T01:12:45.486ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_2217a76b-b58b-4ada-a4c4-e84eca8d38a4") failed: Property not found. retry in 642.591515ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_2217a76b-b58b-4ada-a4c4-e84eca8d38a4
10632023-12-15T01:12:45.506ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_88b23891-1726-4275-973f-1bfd9c7d86da") failed: Property not found. retry in 236.517965ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_88b23891-1726-4275-973f-1bfd9c7d86da
10642023-12-15T01:12:45.529ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_04219130-934c-4126-868a-697ce1f64c38") failed: Property not found. retry in 1.094415434s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_04219130-934c-4126-868a-697ce1f64c38
10652023-12-15T01:12:45.549ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_8b0e0450-abc8-4227-b368-5e4185b52841") failed: Property not found. retry in 1.11430623s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_8b0e0450-abc8-4227-b368-5e4185b52841
10662023-12-15T01:12:45.569ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_5b1f3c9f-bb97-4778-92ff-b6ab4fa6d3f5") failed: Property not found. retry in 60.889585ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_5b1f3c9f-bb97-4778-92ff-b6ab4fa6d3f5
10672023-12-15T01:12:45.584ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_e9d9360e-0097-44f3-aee0-39dd0bc57e9b") failed: Property not found. retry in 101.122104ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_e9d9360e-0097-44f3-aee0-39dd0bc57e9b
10682023-12-15T01:12:45.600ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_1756a188-a654-4c2f-b5bb-e2f18ad4474e") failed: Property not found. retry in 433.311923ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_1756a188-a654-4c2f-b5bb-e2f18ad4474e
10692023-12-15T01:12:45.617ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_17ef7150-35b2-46ab-918a-0b0af0a9b49d") failed: Property not found. retry in 950.252953ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_17ef7150-35b2-46ab-918a-0b0af0a9b49d
10702023-12-15T01:12:45.646ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_5b1f3c9f-bb97-4778-92ff-b6ab4fa6d3f5") failed: Property not found. retry in 78.090561ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_5b1f3c9f-bb97-4778-92ff-b6ab4fa6d3f5
10712023-12-15T01:12:45.671ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_cd0082d8-88ea-4123-9209-754395e437e9") failed: Property not found. retry in 969.495256ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_cd0082d8-88ea-4123-9209-754395e437e9
10722023-12-15T01:12:45.683ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_c710b263-6e30-4f78-bb21-7a9e382a9c52") failed: Property not found. retry in 1.125021032s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_c710b263-6e30-4f78-bb21-7a9e382a9c52
10732023-12-15T01:12:45.700ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_e9d9360e-0097-44f3-aee0-39dd0bc57e9b") failed: Property not found. retry in 385.860633ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_e9d9360e-0097-44f3-aee0-39dd0bc57e9b
10742023-12-15T01:12:45.735ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_5b1f3c9f-bb97-4778-92ff-b6ab4fa6d3f5") failed: Property not found. retry in 190.08652ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_5b1f3c9f-bb97-4778-92ff-b6ab4fa6d3f5
10752023-12-15T01:12:45.745ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_826ad2ba-d609-43a3-9165-9e544e735622") failed: Property not found. retry in 259.424994ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_826ad2ba-d609-43a3-9165-9e544e735622
10762023-12-15T01:12:45.757ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_88b23891-1726-4275-973f-1bfd9c7d86da") failed: Property not found. retry in 389.864366ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_88b23891-1726-4275-973f-1bfd9c7d86da
10772023-12-15T01:12:45.901ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_df48995f-8d48-4496-b8a4-000ccf1c6fe7") failed: Property not found. retry in 566.593947ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_df48995f-8d48-4496-b8a4-000ccf1c6fe7
10782023-12-15T01:12:45.941ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_5b1f3c9f-bb97-4778-92ff-b6ab4fa6d3f5") failed: Property not found. retry in 384.922775ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_5b1f3c9f-bb97-4778-92ff-b6ab4fa6d3f5
10792023-12-15T01:12:46.017ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_826ad2ba-d609-43a3-9165-9e544e735622") failed: Property not found. retry in 510.076392ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_826ad2ba-d609-43a3-9165-9e544e735622
10802023-12-15T01:12:46.047ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_1756a188-a654-4c2f-b5bb-e2f18ad4474e") failed: Property not found. retry in 1.31561183s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_1756a188-a654-4c2f-b5bb-e2f18ad4474e
10812023-12-15T01:12:46.099ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_e9d9360e-0097-44f3-aee0-39dd0bc57e9b") failed: Property not found. retry in 470.351593ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_e9d9360e-0097-44f3-aee0-39dd0bc57e9b
10822023-12-15T01:12:46.140ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_2217a76b-b58b-4ada-a4c4-e84eca8d38a4") failed: Property not found. retry in 1.484531815s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_2217a76b-b58b-4ada-a4c4-e84eca8d38a4
10832023-12-15T01:12:46.160ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_88b23891-1726-4275-973f-1bfd9c7d86da") failed: Property not found. retry in 830.866082ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_88b23891-1726-4275-973f-1bfd9c7d86da
10842023-12-15T01:12:46.205ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_127efcba-5993-4577-bcb2-f4a98a4520bb") failed: Property not found. retry in 1.320703345s file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_127efcba-5993-4577-bcb2-f4a98a4520bb
10852023-12-15T01:12:46.340ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_5b1f3c9f-bb97-4778-92ff-b6ab4fa6d3f5") failed: Property not found. retry in 997.016293ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_5b1f3c9f-bb97-4778-92ff-b6ab4fa6d3f5
10862023-12-15T01:12:46.480ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_df48995f-8d48-4496-b8a4-000ccf1c6fe7") failed: Property not found. retry in 1.038590969s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_df48995f-8d48-4496-b8a4-000ccf1c6fe7
10872023-12-15T01:12:46.540ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_826ad2ba-d609-43a3-9165-9e544e735622") failed: Property not found. retry in 595.761399ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_826ad2ba-d609-43a3-9165-9e544e735622
10882023-12-15T01:12:46.580ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_17ef7150-35b2-46ab-918a-0b0af0a9b49d") failed: Property not found. retry in 658.141771ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_17ef7150-35b2-46ab-918a-0b0af0a9b49d
10892023-12-15T01:12:46.592ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_e9d9360e-0097-44f3-aee0-39dd0bc57e9b") failed: Property not found. retry in 1.288997821s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_e9d9360e-0097-44f3-aee0-39dd0bc57e9b
10902023-12-15T01:12:46.635ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_04219130-934c-4126-868a-697ce1f64c38") failed: Property not found. retry in 794.263873ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_04219130-934c-4126-868a-697ce1f64c38
10912023-12-15T01:12:46.653ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_cd0082d8-88ea-4123-9209-754395e437e9") failed: Property not found. retry in 1.290915621s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_cd0082d8-88ea-4123-9209-754395e437e9
10922023-12-15T01:12:46.676ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_8b0e0450-abc8-4227-b368-5e4185b52841") failed: Property not found. retry in 663.99117ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_8b0e0450-abc8-4227-b368-5e4185b52841
10932023-12-15T01:12:46.814ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_3fd5ef26-c842-4d29-8079-afd0b75a8502") failed: Property not found. retry in 65.494671ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_3fd5ef26-c842-4d29-8079-afd0b75a8502
10942023-12-15T01:12:46.827ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_c710b263-6e30-4f78-bb21-7a9e382a9c52") failed: Property not found. retry in 516.131609ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_c710b263-6e30-4f78-bb21-7a9e382a9c52
10952023-12-15T01:12:46.891ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_3fd5ef26-c842-4d29-8079-afd0b75a8502") failed: Property not found. retry in 106.018691ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_3fd5ef26-c842-4d29-8079-afd0b75a8502
10962023-12-15T01:12:46.907ZINFOSledAgent (ServiceManager): Zone booting file = illumos-utils/src/running_zone.rs:512 zone = oxz_external_dns_326e1e2f-55db-4ad4-aa67-d2bb64a872b8
10972023-12-15T01:12:47.004ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_88b23891-1726-4275-973f-1bfd9c7d86da") failed: Property not found. retry in 609.801966ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_88b23891-1726-4275-973f-1bfd9c7d86da
10982023-12-15T01:12:47.015ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_3fd5ef26-c842-4d29-8079-afd0b75a8502") failed: Property not found. retry in 173.473058ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_3fd5ef26-c842-4d29-8079-afd0b75a8502
10992023-12-15T01:12:47.149ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_826ad2ba-d609-43a3-9165-9e544e735622") failed: Property not found. retry in 1.246781508s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_826ad2ba-d609-43a3-9165-9e544e735622
11002023-12-15T01:12:47.160ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_326e1e2f-55db-4ad4-aa67-d2bb64a872b8") failed: Property not found. retry in 35.600847ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_326e1e2f-55db-4ad4-aa67-d2bb64a872b8
11012023-12-15T01:12:47.204ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_3fd5ef26-c842-4d29-8079-afd0b75a8502") failed: Property not found. retry in 472.094327ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_3fd5ef26-c842-4d29-8079-afd0b75a8502
11022023-12-15T01:12:47.214ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_326e1e2f-55db-4ad4-aa67-d2bb64a872b8") failed: Property not found. retry in 77.347884ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_326e1e2f-55db-4ad4-aa67-d2bb64a872b8
11032023-12-15T01:12:47.251ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_17ef7150-35b2-46ab-918a-0b0af0a9b49d") failed: Property not found. retry in 1.265078403s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_17ef7150-35b2-46ab-918a-0b0af0a9b49d
11042023-12-15T01:12:47.303ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_326e1e2f-55db-4ad4-aa67-d2bb64a872b8") failed: Property not found. retry in 273.07541ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_326e1e2f-55db-4ad4-aa67-d2bb64a872b8
11052023-12-15T01:12:47.348ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_5b1f3c9f-bb97-4778-92ff-b6ab4fa6d3f5") failed: Property not found. retry in 1.251591896s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_5b1f3c9f-bb97-4778-92ff-b6ab4fa6d3f5
11062023-12-15T01:12:47.359ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_8b0e0450-abc8-4227-b368-5e4185b52841") failed: Property not found. retry in 674.749584ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_8b0e0450-abc8-4227-b368-5e4185b52841
11072023-12-15T01:12:47.370ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_c710b263-6e30-4f78-bb21-7a9e382a9c52") failed: Property not found. retry in 842.982845ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_c710b263-6e30-4f78-bb21-7a9e382a9c52
11082023-12-15T01:12:47.381ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_1756a188-a654-4c2f-b5bb-e2f18ad4474e") failed: Property not found. retry in 1.483515112s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_1756a188-a654-4c2f-b5bb-e2f18ad4474e
11092023-12-15T01:12:47.440ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_04219130-934c-4126-868a-697ce1f64c38") failed: Property not found. retry in 840.827636ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_04219130-934c-4126-868a-697ce1f64c38
11102023-12-15T01:12:47.534ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_df48995f-8d48-4496-b8a4-000ccf1c6fe7") failed: Property not found. retry in 1.027675173s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_df48995f-8d48-4496-b8a4-000ccf1c6fe7
11112023-12-15T01:12:47.545ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_127efcba-5993-4577-bcb2-f4a98a4520bb") failed: Property not found. retry in 1.4636111s file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_127efcba-5993-4577-bcb2-f4a98a4520bb
11122023-12-15T01:12:47.589ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_326e1e2f-55db-4ad4-aa67-d2bb64a872b8") failed: Property not found. retry in 447.015196ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_326e1e2f-55db-4ad4-aa67-d2bb64a872b8
11132023-12-15T01:12:47.626ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_88b23891-1726-4275-973f-1bfd9c7d86da") failed: Property not found. retry in 1.141848504s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_88b23891-1726-4275-973f-1bfd9c7d86da
11142023-12-15T01:12:47.636ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_2217a76b-b58b-4ada-a4c4-e84eca8d38a4") failed: Property not found. retry in 951.892469ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_2217a76b-b58b-4ada-a4c4-e84eca8d38a4
11152023-12-15T01:12:47.689ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_3fd5ef26-c842-4d29-8079-afd0b75a8502") failed: Property not found. retry in 523.46022ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_3fd5ef26-c842-4d29-8079-afd0b75a8502
11162023-12-15T01:12:48.126ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_e9d9360e-0097-44f3-aee0-39dd0bc57e9b") failed: Property not found. retry in 1.48859393s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_e9d9360e-0097-44f3-aee0-39dd0bc57e9b
11172023-12-15T01:12:48.182ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_cd0082d8-88ea-4123-9209-754395e437e9") failed: Property not found. retry in 1.423840971s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_cd0082d8-88ea-4123-9209-754395e437e9
11182023-12-15T01:12:48.220ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_8b0e0450-abc8-4227-b368-5e4185b52841") failed: Property not found. retry in 1.414183558s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_8b0e0450-abc8-4227-b368-5e4185b52841
11192023-12-15T01:12:48.253ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_c710b263-6e30-4f78-bb21-7a9e382a9c52") failed: Property not found. retry in 1.337590315s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_c710b263-6e30-4f78-bb21-7a9e382a9c52
11202023-12-15T01:12:48.290ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_326e1e2f-55db-4ad4-aa67-d2bb64a872b8") failed: Property not found. retry in 672.683067ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_326e1e2f-55db-4ad4-aa67-d2bb64a872b8
11212023-12-15T01:12:48.310ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_3fd5ef26-c842-4d29-8079-afd0b75a8502") failed: Property not found. retry in 598.322592ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_3fd5ef26-c842-4d29-8079-afd0b75a8502
11222023-12-15T01:12:48.337ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_04219130-934c-4126-868a-697ce1f64c38") failed: Property not found. retry in 801.719926ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_04219130-934c-4126-868a-697ce1f64c38
11232023-12-15T01:12:48.411ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_826ad2ba-d609-43a3-9165-9e544e735622") failed: Property not found. retry in 937.957588ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_826ad2ba-d609-43a3-9165-9e544e735622
11242023-12-15T01:12:48.477ZINFOSledAgent (ServiceManager): Zone booting file = illumos-utils/src/running_zone.rs:512 zone = oxz_nexus_7198c8dc-786b-4865-a05e-06e4223227c4
11252023-12-15T01:12:48.483ZINFOSledAgent (ServiceManager): Zone booting file = illumos-utils/src/running_zone.rs:512 zone = oxz_nexus_700e7b49-f840-40d5-92e8-83ab758ea8a7
11262023-12-15T01:12:48.529ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_17ef7150-35b2-46ab-918a-0b0af0a9b49d") failed: Property not found. retry in 1.324423531s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_17ef7150-35b2-46ab-918a-0b0af0a9b49d
11272023-12-15T01:12:48.576ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_df48995f-8d48-4496-b8a4-000ccf1c6fe7") failed: Property not found. retry in 1.151194753s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_df48995f-8d48-4496-b8a4-000ccf1c6fe7
11282023-12-15T01:12:48.604ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_2217a76b-b58b-4ada-a4c4-e84eca8d38a4") failed: Property not found. retry in 1.249491775s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_2217a76b-b58b-4ada-a4c4-e84eca8d38a4
11292023-12-15T01:12:48.618ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_5b1f3c9f-bb97-4778-92ff-b6ab4fa6d3f5") failed: Property not found. retry in 1.423855683s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_5b1f3c9f-bb97-4778-92ff-b6ab4fa6d3f5
11302023-12-15T01:12:48.707ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_700e7b49-f840-40d5-92e8-83ab758ea8a7") failed: Property not found. retry in 38.622218ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_700e7b49-f840-40d5-92e8-83ab758ea8a7
11312023-12-15T01:12:48.717ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_7198c8dc-786b-4865-a05e-06e4223227c4") failed: Property not found. retry in 45.655662ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_7198c8dc-786b-4865-a05e-06e4223227c4
11322023-12-15T01:12:48.754ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_700e7b49-f840-40d5-92e8-83ab758ea8a7") failed: Property not found. retry in 127.153516ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_700e7b49-f840-40d5-92e8-83ab758ea8a7
11332023-12-15T01:12:48.773ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_7198c8dc-786b-4865-a05e-06e4223227c4") failed: Property not found. retry in 66.860413ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_7198c8dc-786b-4865-a05e-06e4223227c4
11342023-12-15T01:12:48.784ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_88b23891-1726-4275-973f-1bfd9c7d86da") failed: Property not found. retry in 683.573609ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_88b23891-1726-4275-973f-1bfd9c7d86da
11352023-12-15T01:12:48.851ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_7198c8dc-786b-4865-a05e-06e4223227c4") failed: Property not found. retry in 173.543258ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_7198c8dc-786b-4865-a05e-06e4223227c4
11362023-12-15T01:12:48.877ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_1756a188-a654-4c2f-b5bb-e2f18ad4474e") failed: Property not found. retry in 1.259942934s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_1756a188-a654-4c2f-b5bb-e2f18ad4474e
11372023-12-15T01:12:48.894ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_700e7b49-f840-40d5-92e8-83ab758ea8a7") failed: Property not found. retry in 293.051798ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_700e7b49-f840-40d5-92e8-83ab758ea8a7
11382023-12-15T01:12:48.918ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_3fd5ef26-c842-4d29-8079-afd0b75a8502") failed: Property not found. retry in 967.43785ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_3fd5ef26-c842-4d29-8079-afd0b75a8502
11392023-12-15T01:12:48.972ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_326e1e2f-55db-4ad4-aa67-d2bb64a872b8") failed: Property not found. retry in 1.408980192s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_326e1e2f-55db-4ad4-aa67-d2bb64a872b8
11402023-12-15T01:12:49.020ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_127efcba-5993-4577-bcb2-f4a98a4520bb") failed: Property not found. retry in 1.279179914s file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_127efcba-5993-4577-bcb2-f4a98a4520bb
11412023-12-15T01:12:49.037ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_7198c8dc-786b-4865-a05e-06e4223227c4") failed: Property not found. retry in 570.070973ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_7198c8dc-786b-4865-a05e-06e4223227c4
11422023-12-15T01:12:49.042ZINFOSledAgent (ServiceManager): Profile for oxz_clickhouse_23e76ebf-1083-415c-b1cc-78b70ea0ee28: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/clickhouse"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value="oxControlService20"/> <propval type="astring" name="gateway" value="fd00:1122:3344:101::1"/> <propval type="astring" name="listen_addr" value="fd00:1122:3344:101::e"/> <propval type="astring" name="listen_port" value="8123"/> <propval type="astring" name="store" value="/data"/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/install"> <property_group type="application" name="install_props"> <property type="net_address" name="nameserver"> <net_address_list> <value_node value="fd00:1122:3344:2::1" /> <value_node value="fd00:1122:3344:1::1" /> <value_node value="fd00:1122:3344:3::1" /> </net_address_list> </property> </property_group> <instance enabled="true" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
11432023-12-15T01:12:49.043ZINFOSledAgent (ServiceManager): Zone booting file = illumos-utils/src/running_zone.rs:512 zone = oxz_clickhouse_23e76ebf-1083-415c-b1cc-78b70ea0ee28
11442023-12-15T01:12:49.151ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_04219130-934c-4126-868a-697ce1f64c38") failed: Property not found. retry in 1.01994061s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_04219130-934c-4126-868a-697ce1f64c38
11452023-12-15T01:12:49.205ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_700e7b49-f840-40d5-92e8-83ab758ea8a7") failed: Property not found. retry in 563.271103ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_700e7b49-f840-40d5-92e8-83ab758ea8a7
11462023-12-15T01:12:49.258ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_23e76ebf-1083-415c-b1cc-78b70ea0ee28") failed: Property not found. retry in 29.417831ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_23e76ebf-1083-415c-b1cc-78b70ea0ee28
11472023-12-15T01:12:49.299ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_23e76ebf-1083-415c-b1cc-78b70ea0ee28") failed: Property not found. retry in 68.761495ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_23e76ebf-1083-415c-b1cc-78b70ea0ee28
11482023-12-15T01:12:49.360ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_826ad2ba-d609-43a3-9165-9e544e735622") failed: Property not found. retry in 868.454443ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_826ad2ba-d609-43a3-9165-9e544e735622
11492023-12-15T01:12:49.378ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_23e76ebf-1083-415c-b1cc-78b70ea0ee28") failed: Property not found. retry in 252.447783ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_23e76ebf-1083-415c-b1cc-78b70ea0ee28
11502023-12-15T01:12:49.478ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_88b23891-1726-4275-973f-1bfd9c7d86da") failed: Property not found. retry in 1.237036866s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_88b23891-1726-4275-973f-1bfd9c7d86da
11512023-12-15T01:12:49.515ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::605a:15ff:fea0:c066": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
11522023-12-15T01:12:49.604ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_c710b263-6e30-4f78-bb21-7a9e382a9c52") failed: Property not found. retry in 991.146663ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_c710b263-6e30-4f78-bb21-7a9e382a9c52
11532023-12-15T01:12:49.616ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_cd0082d8-88ea-4123-9209-754395e437e9") failed: Property not found. retry in 763.218657ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_cd0082d8-88ea-4123-9209-754395e437e9
11542023-12-15T01:12:49.628ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_7198c8dc-786b-4865-a05e-06e4223227c4") failed: Property not found. retry in 1.126928025s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_7198c8dc-786b-4865-a05e-06e4223227c4
11552023-12-15T01:12:49.637ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_e9d9360e-0097-44f3-aee0-39dd0bc57e9b") failed: Property not found. retry in 1.07499591s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_e9d9360e-0097-44f3-aee0-39dd0bc57e9b
11562023-12-15T01:12:49.646ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_8b0e0450-abc8-4227-b368-5e4185b52841") failed: Property not found. retry in 1.230158759s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_8b0e0450-abc8-4227-b368-5e4185b52841
11572023-12-15T01:12:49.655ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_23e76ebf-1083-415c-b1cc-78b70ea0ee28") failed: Property not found. retry in 247.883743ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_23e76ebf-1083-415c-b1cc-78b70ea0ee28
11582023-12-15T01:12:49.738ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_df48995f-8d48-4496-b8a4-000ccf1c6fe7") failed: Property not found. retry in 750.837794ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_df48995f-8d48-4496-b8a4-000ccf1c6fe7
11592023-12-15T01:12:49.778ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_700e7b49-f840-40d5-92e8-83ab758ea8a7") failed: Property not found. retry in 1.05815751s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_700e7b49-f840-40d5-92e8-83ab758ea8a7
11602023-12-15T01:12:49.865ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_2217a76b-b58b-4ada-a4c4-e84eca8d38a4") failed: Property not found. retry in 1.437390917s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_2217a76b-b58b-4ada-a4c4-e84eca8d38a4
11612023-12-15T01:12:49.875ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_17ef7150-35b2-46ab-918a-0b0af0a9b49d") failed: Property not found. retry in 1.086998423s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_17ef7150-35b2-46ab-918a-0b0af0a9b49d
11622023-12-15T01:12:49.896ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_3fd5ef26-c842-4d29-8079-afd0b75a8502") failed: Property not found. retry in 865.808296ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_3fd5ef26-c842-4d29-8079-afd0b75a8502
11632023-12-15T01:12:49.914ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_23e76ebf-1083-415c-b1cc-78b70ea0ee28") failed: Property not found. retry in 741.042292ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_23e76ebf-1083-415c-b1cc-78b70ea0ee28
11642023-12-15T01:12:50.052ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_5b1f3c9f-bb97-4778-92ff-b6ab4fa6d3f5") failed: Property not found. retry in 1.235377261s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_5b1f3c9f-bb97-4778-92ff-b6ab4fa6d3f5
11652023-12-15T01:12:50.151ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_1756a188-a654-4c2f-b5bb-e2f18ad4474e") failed: Property not found. retry in 637.297873ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_1756a188-a654-4c2f-b5bb-e2f18ad4474e
11662023-12-15T01:12:50.186ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_04219130-934c-4126-868a-697ce1f64c38") failed: Property not found. retry in 1.067893003s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_04219130-934c-4126-868a-697ce1f64c38
11672023-12-15T01:12:50.242ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_826ad2ba-d609-43a3-9165-9e544e735622") failed: Property not found. retry in 573.953471ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_826ad2ba-d609-43a3-9165-9e544e735622
11682023-12-15T01:12:50.311ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_127efcba-5993-4577-bcb2-f4a98a4520bb") failed: Property not found. retry in 1.293078629s file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_127efcba-5993-4577-bcb2-f4a98a4520bb
11692023-12-15T01:12:50.499ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_cd0082d8-88ea-4123-9209-754395e437e9") failed: Property not found. retry in 1.44970729s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_cd0082d8-88ea-4123-9209-754395e437e9
11702023-12-15T01:12:50.516ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_df48995f-8d48-4496-b8a4-000ccf1c6fe7") failed: Property not found. retry in 691.325626ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_df48995f-8d48-4496-b8a4-000ccf1c6fe7
11712023-12-15T01:12:50.556ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_326e1e2f-55db-4ad4-aa67-d2bb64a872b8") failed: Property not found. retry in 1.122929786s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_326e1e2f-55db-4ad4-aa67-d2bb64a872b8
11722023-12-15T01:12:50.614ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_c710b263-6e30-4f78-bb21-7a9e382a9c52") failed: Property not found. retry in 746.92559ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_c710b263-6e30-4f78-bb21-7a9e382a9c52
11732023-12-15T01:12:50.670ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_23e76ebf-1083-415c-b1cc-78b70ea0ee28") failed: Property not found. retry in 1.364487446s file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_23e76ebf-1083-415c-b1cc-78b70ea0ee28
11742023-12-15T01:12:50.725ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_e9d9360e-0097-44f3-aee0-39dd0bc57e9b") failed: Property not found. retry in 567.012127ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_e9d9360e-0097-44f3-aee0-39dd0bc57e9b
11752023-12-15T01:12:50.738ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_88b23891-1726-4275-973f-1bfd9c7d86da") failed: Property not found. retry in 1.210510574s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_88b23891-1726-4275-973f-1bfd9c7d86da
11762023-12-15T01:12:50.768ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_7198c8dc-786b-4865-a05e-06e4223227c4") failed: Property not found. retry in 754.862809ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_7198c8dc-786b-4865-a05e-06e4223227c4
11772023-12-15T01:12:50.781ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_3fd5ef26-c842-4d29-8079-afd0b75a8502") failed: Property not found. retry in 1.191022324s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_3fd5ef26-c842-4d29-8079-afd0b75a8502
11782023-12-15T01:12:50.801ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_1756a188-a654-4c2f-b5bb-e2f18ad4474e") failed: Property not found. retry in 1.133936052s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_1756a188-a654-4c2f-b5bb-e2f18ad4474e
11792023-12-15T01:12:50.828ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_826ad2ba-d609-43a3-9165-9e544e735622") failed: Property not found. retry in 1.29298251s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_826ad2ba-d609-43a3-9165-9e544e735622
11802023-12-15T01:12:50.848ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_700e7b49-f840-40d5-92e8-83ab758ea8a7") failed: Property not found. retry in 583.781147ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_700e7b49-f840-40d5-92e8-83ab758ea8a7
11812023-12-15T01:12:50.857ZINFOSledAgent (ServiceManager): Zone booting file = illumos-utils/src/running_zone.rs:512 zone = oxz_nexus_72d8f37c-a72e-4800-9c47-39f6b4e0d24b
11822023-12-15T01:12:50.889ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_8b0e0450-abc8-4227-b368-5e4185b52841") failed: Property not found. retry in 1.268332744s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_8b0e0450-abc8-4227-b368-5e4185b52841
11832023-12-15T01:12:50.975ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_17ef7150-35b2-46ab-918a-0b0af0a9b49d") failed: Property not found. retry in 1.122746888s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_17ef7150-35b2-46ab-918a-0b0af0a9b49d
11842023-12-15T01:12:51.093ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_72d8f37c-a72e-4800-9c47-39f6b4e0d24b") failed: Property not found. retry in 41.358684ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_72d8f37c-a72e-4800-9c47-39f6b4e0d24b
11852023-12-15T01:12:51.145ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_72d8f37c-a72e-4800-9c47-39f6b4e0d24b") failed: Property not found. retry in 145.313313ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_72d8f37c-a72e-4800-9c47-39f6b4e0d24b
11862023-12-15T01:12:51.227ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_df48995f-8d48-4496-b8a4-000ccf1c6fe7") failed: Property not found. retry in 1.263424872s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_df48995f-8d48-4496-b8a4-000ccf1c6fe7
11872023-12-15T01:12:51.267ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_04219130-934c-4126-868a-697ce1f64c38") failed: Property not found. retry in 1.108250107s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_04219130-934c-4126-868a-697ce1f64c38
11882023-12-15T01:12:51.300ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_5b1f3c9f-bb97-4778-92ff-b6ab4fa6d3f5") failed: Property not found. retry in 805.646312ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_5b1f3c9f-bb97-4778-92ff-b6ab4fa6d3f5
11892023-12-15T01:12:51.310ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_72d8f37c-a72e-4800-9c47-39f6b4e0d24b") failed: Property not found. retry in 266.950596ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_72d8f37c-a72e-4800-9c47-39f6b4e0d24b
11902023-12-15T01:12:51.322ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_e9d9360e-0097-44f3-aee0-39dd0bc57e9b") failed: Property not found. retry in 1.455963913s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_e9d9360e-0097-44f3-aee0-39dd0bc57e9b
11912023-12-15T01:12:51.333ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_2217a76b-b58b-4ada-a4c4-e84eca8d38a4") failed: Property not found. retry in 1.473990123s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_2217a76b-b58b-4ada-a4c4-e84eca8d38a4
11922023-12-15T01:12:51.373ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_c710b263-6e30-4f78-bb21-7a9e382a9c52") failed: Property not found. retry in 1.367031196s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_c710b263-6e30-4f78-bb21-7a9e382a9c52
11932023-12-15T01:12:51.443ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_700e7b49-f840-40d5-92e8-83ab758ea8a7") failed: Property not found. retry in 907.690382ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_700e7b49-f840-40d5-92e8-83ab758ea8a7
11942023-12-15T01:12:51.536ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_7198c8dc-786b-4865-a05e-06e4223227c4") failed: Property not found. retry in 945.915959ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_7198c8dc-786b-4865-a05e-06e4223227c4
11952023-12-15T01:12:51.591ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_72d8f37c-a72e-4800-9c47-39f6b4e0d24b") failed: Property not found. retry in 332.509738ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_72d8f37c-a72e-4800-9c47-39f6b4e0d24b
11962023-12-15T01:12:51.617ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_127efcba-5993-4577-bcb2-f4a98a4520bb") failed: Property not found. retry in 1.386713834s file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_127efcba-5993-4577-bcb2-f4a98a4520bb
11972023-12-15T01:12:51.692ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_326e1e2f-55db-4ad4-aa67-d2bb64a872b8") failed: Property not found. retry in 651.510668ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_326e1e2f-55db-4ad4-aa67-d2bb64a872b8
11982023-12-15T01:12:51.936ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_72d8f37c-a72e-4800-9c47-39f6b4e0d24b") failed: Property not found. retry in 471.67026ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_72d8f37c-a72e-4800-9c47-39f6b4e0d24b
11992023-12-15T01:12:51.946ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_1756a188-a654-4c2f-b5bb-e2f18ad4474e") failed: Property not found. retry in 1.191120767s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_1756a188-a654-4c2f-b5bb-e2f18ad4474e
12002023-12-15T01:12:51.960ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_cd0082d8-88ea-4123-9209-754395e437e9") failed: Property not found. retry in 670.781368ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_cd0082d8-88ea-4123-9209-754395e437e9
12012023-12-15T01:12:51.971ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_88b23891-1726-4275-973f-1bfd9c7d86da") failed: Property not found. retry in 696.697692ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_88b23891-1726-4275-973f-1bfd9c7d86da
12022023-12-15T01:12:51.982ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_3fd5ef26-c842-4d29-8079-afd0b75a8502") failed: Property not found. retry in 1.213048475s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_3fd5ef26-c842-4d29-8079-afd0b75a8502
12032023-12-15T01:12:52.048ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_23e76ebf-1083-415c-b1cc-78b70ea0ee28") failed: Property not found. retry in 1.104958487s file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_23e76ebf-1083-415c-b1cc-78b70ea0ee28
12042023-12-15T01:12:52.110ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_17ef7150-35b2-46ab-918a-0b0af0a9b49d") failed: Property not found. retry in 984.890928ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_17ef7150-35b2-46ab-918a-0b0af0a9b49d
12052023-12-15T01:12:52.122ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_5b1f3c9f-bb97-4778-92ff-b6ab4fa6d3f5") failed: Property not found. retry in 1.447904546s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_5b1f3c9f-bb97-4778-92ff-b6ab4fa6d3f5
12062023-12-15T01:12:52.133ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_826ad2ba-d609-43a3-9165-9e544e735622") failed: Property not found. retry in 862.01197ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_826ad2ba-d609-43a3-9165-9e544e735622
12072023-12-15T01:12:52.169ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_8b0e0450-abc8-4227-b368-5e4185b52841") failed: Property not found. retry in 1.267045402s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_8b0e0450-abc8-4227-b368-5e4185b52841
12082023-12-15T01:12:52.356ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_326e1e2f-55db-4ad4-aa67-d2bb64a872b8") failed: Property not found. retry in 971.01974ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_326e1e2f-55db-4ad4-aa67-d2bb64a872b8
12092023-12-15T01:12:52.367ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_700e7b49-f840-40d5-92e8-83ab758ea8a7") failed: Property not found. retry in 1.067195818s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_700e7b49-f840-40d5-92e8-83ab758ea8a7
12102023-12-15T01:12:52.387ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_04219130-934c-4126-868a-697ce1f64c38") failed: Property not found. retry in 816.525838ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_04219130-934c-4126-868a-697ce1f64c38
12112023-12-15T01:12:52.423ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_72d8f37c-a72e-4800-9c47-39f6b4e0d24b") failed: Property not found. retry in 878.354708ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_72d8f37c-a72e-4800-9c47-39f6b4e0d24b
12122023-12-15T01:12:52.494ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_7198c8dc-786b-4865-a05e-06e4223227c4") failed: Property not found. retry in 1.331708398s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_7198c8dc-786b-4865-a05e-06e4223227c4
12132023-12-15T01:12:52.505ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_df48995f-8d48-4496-b8a4-000ccf1c6fe7") failed: Property not found. retry in 507.421312ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_df48995f-8d48-4496-b8a4-000ccf1c6fe7
12142023-12-15T01:12:52.644ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_cd0082d8-88ea-4123-9209-754395e437e9") failed: Property not found. retry in 891.206106ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_cd0082d8-88ea-4123-9209-754395e437e9
12152023-12-15T01:12:52.683ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_88b23891-1726-4275-973f-1bfd9c7d86da") failed: Property not found. retry in 1.420480152s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_88b23891-1726-4275-973f-1bfd9c7d86da
12162023-12-15T01:12:52.754ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_c710b263-6e30-4f78-bb21-7a9e382a9c52") failed: Property not found. retry in 785.628956ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_c710b263-6e30-4f78-bb21-7a9e382a9c52
12172023-12-15T01:12:52.792ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_e9d9360e-0097-44f3-aee0-39dd0bc57e9b") failed: Property not found. retry in 813.485715ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_e9d9360e-0097-44f3-aee0-39dd0bc57e9b
12182023-12-15T01:12:52.822ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_2217a76b-b58b-4ada-a4c4-e84eca8d38a4") failed: Property not found. retry in 581.088843ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_2217a76b-b58b-4ada-a4c4-e84eca8d38a4
12192023-12-15T01:12:53.009ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_826ad2ba-d609-43a3-9165-9e544e735622") failed: Property not found. retry in 974.531222ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_826ad2ba-d609-43a3-9165-9e544e735622
12202023-12-15T01:12:53.022ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_127efcba-5993-4577-bcb2-f4a98a4520bb") failed: Property not found. retry in 557.502169ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_127efcba-5993-4577-bcb2-f4a98a4520bb
12212023-12-15T01:12:53.033ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_df48995f-8d48-4496-b8a4-000ccf1c6fe7") failed: Property not found. retry in 1.356417336s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_df48995f-8d48-4496-b8a4-000ccf1c6fe7
12222023-12-15T01:12:53.107ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_17ef7150-35b2-46ab-918a-0b0af0a9b49d") failed: Property not found. retry in 1.373935221s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_17ef7150-35b2-46ab-918a-0b0af0a9b49d
12232023-12-15T01:12:53.150ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_1756a188-a654-4c2f-b5bb-e2f18ad4474e") failed: Property not found. retry in 1.105427775s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_1756a188-a654-4c2f-b5bb-e2f18ad4474e
12242023-12-15T01:12:53.166ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_23e76ebf-1083-415c-b1cc-78b70ea0ee28") failed: Property not found. retry in 541.876283ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_23e76ebf-1083-415c-b1cc-78b70ea0ee28
12252023-12-15T01:12:53.209ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_3fd5ef26-c842-4d29-8079-afd0b75a8502") failed: Property not found. retry in 581.840087ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_3fd5ef26-c842-4d29-8079-afd0b75a8502
12262023-12-15T01:12:53.221ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_04219130-934c-4126-868a-697ce1f64c38") failed: Property not found. retry in 957.862903ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_04219130-934c-4126-868a-697ce1f64c38
12272023-12-15T01:12:53.313ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_72d8f37c-a72e-4800-9c47-39f6b4e0d24b") failed: Property not found. retry in 1.341851477s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_72d8f37c-a72e-4800-9c47-39f6b4e0d24b
12282023-12-15T01:12:53.339ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_326e1e2f-55db-4ad4-aa67-d2bb64a872b8") failed: Property not found. retry in 925.683858ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_326e1e2f-55db-4ad4-aa67-d2bb64a872b8
12292023-12-15T01:12:53.415ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_2217a76b-b58b-4ada-a4c4-e84eca8d38a4") failed: Property not found. retry in 1.134553128s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_2217a76b-b58b-4ada-a4c4-e84eca8d38a4
12302023-12-15T01:12:53.446ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_700e7b49-f840-40d5-92e8-83ab758ea8a7") failed: Property not found. retry in 1.185916519s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_700e7b49-f840-40d5-92e8-83ab758ea8a7
12312023-12-15T01:12:53.456ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_8b0e0450-abc8-4227-b368-5e4185b52841") failed: Property not found. retry in 1.076715369s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_8b0e0450-abc8-4227-b368-5e4185b52841
12322023-12-15T01:12:53.548ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_cd0082d8-88ea-4123-9209-754395e437e9") failed: Property not found. retry in 977.114777ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_cd0082d8-88ea-4123-9209-754395e437e9
12332023-12-15T01:12:53.562ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_c710b263-6e30-4f78-bb21-7a9e382a9c52") failed: Property not found. retry in 944.685928ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_c710b263-6e30-4f78-bb21-7a9e382a9c52
12342023-12-15T01:12:53.590ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_5b1f3c9f-bb97-4778-92ff-b6ab4fa6d3f5") failed: Property not found. retry in 1.083006022s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_5b1f3c9f-bb97-4778-92ff-b6ab4fa6d3f5
12352023-12-15T01:12:53.606ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_127efcba-5993-4577-bcb2-f4a98a4520bb") failed: Property not found. retry in 1.392609355s file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_127efcba-5993-4577-bcb2-f4a98a4520bb
12362023-12-15T01:12:53.624ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_e9d9360e-0097-44f3-aee0-39dd0bc57e9b") failed: Property not found. retry in 1.082412576s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_e9d9360e-0097-44f3-aee0-39dd0bc57e9b
12372023-12-15T01:12:53.724ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_23e76ebf-1083-415c-b1cc-78b70ea0ee28") failed: Property not found. retry in 1.10689723s file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_23e76ebf-1083-415c-b1cc-78b70ea0ee28
12382023-12-15T01:12:53.805ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_3fd5ef26-c842-4d29-8079-afd0b75a8502") failed: Property not found. retry in 809.643506ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_3fd5ef26-c842-4d29-8079-afd0b75a8502
12392023-12-15T01:12:53.841ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_7198c8dc-786b-4865-a05e-06e4223227c4") failed: Property not found. retry in 1.147262786s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_7198c8dc-786b-4865-a05e-06e4223227c4
12402023-12-15T01:12:54.004ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_826ad2ba-d609-43a3-9165-9e544e735622") failed: Property not found. retry in 700.480148ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_826ad2ba-d609-43a3-9165-9e544e735622
12412023-12-15T01:12:54.120ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_88b23891-1726-4275-973f-1bfd9c7d86da") failed: Property not found. retry in 1.353529772s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_88b23891-1726-4275-973f-1bfd9c7d86da
12422023-12-15T01:12:54.197ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_04219130-934c-4126-868a-697ce1f64c38") failed: Property not found. retry in 1.097414785s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_04219130-934c-4126-868a-697ce1f64c38
12432023-12-15T01:12:54.273ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_1756a188-a654-4c2f-b5bb-e2f18ad4474e") failed: Property not found. retry in 1.091705529s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_1756a188-a654-4c2f-b5bb-e2f18ad4474e
12442023-12-15T01:12:54.290ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_326e1e2f-55db-4ad4-aa67-d2bb64a872b8") failed: Property not found. retry in 606.550733ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_326e1e2f-55db-4ad4-aa67-d2bb64a872b8
12452023-12-15T01:12:54.406ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_df48995f-8d48-4496-b8a4-000ccf1c6fe7") failed: Property not found. retry in 1.043117865s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_df48995f-8d48-4496-b8a4-000ccf1c6fe7
12462023-12-15T01:12:54.496ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_17ef7150-35b2-46ab-918a-0b0af0a9b49d") failed: Property not found. retry in 1.2222592s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_17ef7150-35b2-46ab-918a-0b0af0a9b49d
12472023-12-15T01:12:54.516ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::605a:15ff:fea0:c066": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
12482023-12-15T01:12:54.525ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_c710b263-6e30-4f78-bb21-7a9e382a9c52") failed: Property not found. retry in 964.982838ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_c710b263-6e30-4f78-bb21-7a9e382a9c52
12492023-12-15T01:12:54.543ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_cd0082d8-88ea-4123-9209-754395e437e9") failed: Property not found. retry in 857.808582ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_cd0082d8-88ea-4123-9209-754395e437e9
12502023-12-15T01:12:54.555ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_8b0e0450-abc8-4227-b368-5e4185b52841") failed: Property not found. retry in 1.474203588s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_8b0e0450-abc8-4227-b368-5e4185b52841
12512023-12-15T01:12:54.568ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_2217a76b-b58b-4ada-a4c4-e84eca8d38a4") failed: Property not found. retry in 918.528955ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_2217a76b-b58b-4ada-a4c4-e84eca8d38a4
12522023-12-15T01:12:54.627ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_3fd5ef26-c842-4d29-8079-afd0b75a8502") failed: Property not found. retry in 1.153672745s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_3fd5ef26-c842-4d29-8079-afd0b75a8502
12532023-12-15T01:12:54.645ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_700e7b49-f840-40d5-92e8-83ab758ea8a7") failed: Property not found. retry in 698.055613ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_700e7b49-f840-40d5-92e8-83ab758ea8a7
12542023-12-15T01:12:54.667ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_72d8f37c-a72e-4800-9c47-39f6b4e0d24b") failed: Property not found. retry in 816.743955ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_72d8f37c-a72e-4800-9c47-39f6b4e0d24b
12552023-12-15T01:12:54.684ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_5b1f3c9f-bb97-4778-92ff-b6ab4fa6d3f5") failed: Property not found. retry in 1.216773553s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_5b1f3c9f-bb97-4778-92ff-b6ab4fa6d3f5
12562023-12-15T01:12:54.718ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_826ad2ba-d609-43a3-9165-9e544e735622") failed: Property not found. retry in 1.372067681s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_826ad2ba-d609-43a3-9165-9e544e735622
12572023-12-15T01:12:54.731ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_e9d9360e-0097-44f3-aee0-39dd0bc57e9b") failed: Property not found. retry in 612.11261ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_e9d9360e-0097-44f3-aee0-39dd0bc57e9b
12582023-12-15T01:12:54.845ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_23e76ebf-1083-415c-b1cc-78b70ea0ee28") failed: Property not found. retry in 1.41624646s file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_23e76ebf-1083-415c-b1cc-78b70ea0ee28
12592023-12-15T01:12:54.909ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_326e1e2f-55db-4ad4-aa67-d2bb64a872b8") failed: Property not found. retry in 753.253142ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_326e1e2f-55db-4ad4-aa67-d2bb64a872b8
12602023-12-15T01:12:55.002ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_7198c8dc-786b-4865-a05e-06e4223227c4") failed: Property not found. retry in 704.916346ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_7198c8dc-786b-4865-a05e-06e4223227c4
12612023-12-15T01:12:55.014ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_127efcba-5993-4577-bcb2-f4a98a4520bb") failed: Property not found. retry in 945.794975ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_127efcba-5993-4577-bcb2-f4a98a4520bb
12622023-12-15T01:12:55.309ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_04219130-934c-4126-868a-697ce1f64c38") failed: Property not found. retry in 567.87941ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_04219130-934c-4126-868a-697ce1f64c38
12632023-12-15T01:12:55.357ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_e9d9360e-0097-44f3-aee0-39dd0bc57e9b") failed: Property not found. retry in 1.334635555s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_e9d9360e-0097-44f3-aee0-39dd0bc57e9b
12642023-12-15T01:12:55.381ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_700e7b49-f840-40d5-92e8-83ab758ea8a7") failed: Property not found. retry in 610.915384ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_700e7b49-f840-40d5-92e8-83ab758ea8a7
12652023-12-15T01:12:55.397ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_1756a188-a654-4c2f-b5bb-e2f18ad4474e") failed: Property not found. retry in 1.208990845s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_1756a188-a654-4c2f-b5bb-e2f18ad4474e
12662023-12-15T01:12:55.419ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_cd0082d8-88ea-4123-9209-754395e437e9") failed: Property not found. retry in 1.333655033s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_cd0082d8-88ea-4123-9209-754395e437e9
12672023-12-15T01:12:55.463ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_df48995f-8d48-4496-b8a4-000ccf1c6fe7") failed: Property not found. retry in 1.358359466s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_df48995f-8d48-4496-b8a4-000ccf1c6fe7
12682023-12-15T01:12:55.487ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_88b23891-1726-4275-973f-1bfd9c7d86da") failed: Property not found. retry in 727.710409ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_88b23891-1726-4275-973f-1bfd9c7d86da
12692023-12-15T01:12:55.499ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_72d8f37c-a72e-4800-9c47-39f6b4e0d24b") failed: Property not found. retry in 797.363947ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_72d8f37c-a72e-4800-9c47-39f6b4e0d24b
12702023-12-15T01:12:55.512ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_2217a76b-b58b-4ada-a4c4-e84eca8d38a4") failed: Property not found. retry in 1.109027339s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_2217a76b-b58b-4ada-a4c4-e84eca8d38a4
12712023-12-15T01:12:55.522ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_c710b263-6e30-4f78-bb21-7a9e382a9c52") failed: Property not found. retry in 1.327791158s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_c710b263-6e30-4f78-bb21-7a9e382a9c52
12722023-12-15T01:12:55.676ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_326e1e2f-55db-4ad4-aa67-d2bb64a872b8") failed: Property not found. retry in 601.812578ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_326e1e2f-55db-4ad4-aa67-d2bb64a872b8
12732023-12-15T01:12:55.717ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_7198c8dc-786b-4865-a05e-06e4223227c4") failed: Property not found. retry in 885.950338ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_7198c8dc-786b-4865-a05e-06e4223227c4
12742023-12-15T01:12:55.730ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_17ef7150-35b2-46ab-918a-0b0af0a9b49d") failed: Property not found. retry in 1.217860575s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_17ef7150-35b2-46ab-918a-0b0af0a9b49d
12752023-12-15T01:12:55.793ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_3fd5ef26-c842-4d29-8079-afd0b75a8502") failed: Property not found. retry in 993.961039ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_3fd5ef26-c842-4d29-8079-afd0b75a8502
12762023-12-15T01:12:55.889ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_04219130-934c-4126-868a-697ce1f64c38") failed: Property not found. retry in 626.203048ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_04219130-934c-4126-868a-697ce1f64c38
12772023-12-15T01:12:55.911ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_5b1f3c9f-bb97-4778-92ff-b6ab4fa6d3f5") failed: Property not found. retry in 1.160805281s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_5b1f3c9f-bb97-4778-92ff-b6ab4fa6d3f5
12782023-12-15T01:12:55.969ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_127efcba-5993-4577-bcb2-f4a98a4520bb") failed: Property not found. retry in 985.124569ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_127efcba-5993-4577-bcb2-f4a98a4520bb
12792023-12-15T01:12:56.002ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_700e7b49-f840-40d5-92e8-83ab758ea8a7") failed: Property not found. retry in 1.273545817s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_700e7b49-f840-40d5-92e8-83ab758ea8a7
12802023-12-15T01:12:56.041ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_8b0e0450-abc8-4227-b368-5e4185b52841") failed: Property not found. retry in 1.309947206s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_8b0e0450-abc8-4227-b368-5e4185b52841
12812023-12-15T01:12:56.102ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_826ad2ba-d609-43a3-9165-9e544e735622") failed: Property not found. retry in 1.237982289s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_826ad2ba-d609-43a3-9165-9e544e735622
12822023-12-15T01:12:56.225ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_88b23891-1726-4275-973f-1bfd9c7d86da") failed: Property not found. retry in 605.311804ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_88b23891-1726-4275-973f-1bfd9c7d86da
12832023-12-15T01:12:56.272ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_23e76ebf-1083-415c-b1cc-78b70ea0ee28") failed: Property not found. retry in 739.399975ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_23e76ebf-1083-415c-b1cc-78b70ea0ee28
12842023-12-15T01:12:56.287ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_326e1e2f-55db-4ad4-aa67-d2bb64a872b8") failed: Property not found. retry in 725.638198ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_326e1e2f-55db-4ad4-aa67-d2bb64a872b8
12852023-12-15T01:12:56.308ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_72d8f37c-a72e-4800-9c47-39f6b4e0d24b") failed: Property not found. retry in 993.675368ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_72d8f37c-a72e-4800-9c47-39f6b4e0d24b
12862023-12-15T01:12:56.527ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_04219130-934c-4126-868a-697ce1f64c38") failed: Property not found. retry in 1.147406855s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_04219130-934c-4126-868a-697ce1f64c38
12872023-12-15T01:12:56.618ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_7198c8dc-786b-4865-a05e-06e4223227c4") failed: Property not found. retry in 754.762354ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_7198c8dc-786b-4865-a05e-06e4223227c4
12882023-12-15T01:12:56.633ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_1756a188-a654-4c2f-b5bb-e2f18ad4474e") failed: Property not found. retry in 697.047758ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_1756a188-a654-4c2f-b5bb-e2f18ad4474e
12892023-12-15T01:12:56.646ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_2217a76b-b58b-4ada-a4c4-e84eca8d38a4") failed: Property not found. retry in 1.055498493s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_2217a76b-b58b-4ada-a4c4-e84eca8d38a4
12902023-12-15T01:12:56.706ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_e9d9360e-0097-44f3-aee0-39dd0bc57e9b") failed: Property not found. retry in 1.368242448s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_e9d9360e-0097-44f3-aee0-39dd0bc57e9b
12912023-12-15T01:12:56.767ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_cd0082d8-88ea-4123-9209-754395e437e9") failed: Property not found. retry in 939.069001ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_cd0082d8-88ea-4123-9209-754395e437e9
12922023-12-15T01:12:56.801ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_3fd5ef26-c842-4d29-8079-afd0b75a8502") failed: Property not found. retry in 1.156616927s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_3fd5ef26-c842-4d29-8079-afd0b75a8502
12932023-12-15T01:12:56.835ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_df48995f-8d48-4496-b8a4-000ccf1c6fe7") failed: Property not found. retry in 797.241164ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_df48995f-8d48-4496-b8a4-000ccf1c6fe7
12942023-12-15T01:12:56.846ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_88b23891-1726-4275-973f-1bfd9c7d86da") failed: Property not found. retry in 1.013075673s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_88b23891-1726-4275-973f-1bfd9c7d86da
12952023-12-15T01:12:56.863ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_c710b263-6e30-4f78-bb21-7a9e382a9c52") failed: Property not found. retry in 661.851476ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_c710b263-6e30-4f78-bb21-7a9e382a9c52
12962023-12-15T01:12:56.963ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_17ef7150-35b2-46ab-918a-0b0af0a9b49d") failed: Property not found. retry in 1.49035406s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_17ef7150-35b2-46ab-918a-0b0af0a9b49d
12972023-12-15T01:12:56.975ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_127efcba-5993-4577-bcb2-f4a98a4520bb") failed: Property not found. retry in 1.145694515s file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_127efcba-5993-4577-bcb2-f4a98a4520bb
12982023-12-15T01:12:57.030ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_23e76ebf-1083-415c-b1cc-78b70ea0ee28") failed: Property not found. retry in 846.635246ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_23e76ebf-1083-415c-b1cc-78b70ea0ee28
12992023-12-15T01:12:57.045ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_326e1e2f-55db-4ad4-aa67-d2bb64a872b8") failed: Property not found. retry in 1.153401632s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_326e1e2f-55db-4ad4-aa67-d2bb64a872b8
13002023-12-15T01:12:57.085ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_5b1f3c9f-bb97-4778-92ff-b6ab4fa6d3f5") failed: Property not found. retry in 1.473473056s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_5b1f3c9f-bb97-4778-92ff-b6ab4fa6d3f5
13012023-12-15T01:12:57.293ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_700e7b49-f840-40d5-92e8-83ab758ea8a7") failed: Property not found. retry in 1.086260333s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_700e7b49-f840-40d5-92e8-83ab758ea8a7
13022023-12-15T01:12:57.315ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_72d8f37c-a72e-4800-9c47-39f6b4e0d24b") failed: Property not found. retry in 906.48211ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_72d8f37c-a72e-4800-9c47-39f6b4e0d24b
13032023-12-15T01:12:57.345ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_1756a188-a654-4c2f-b5bb-e2f18ad4474e") failed: Property not found. retry in 887.592762ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_1756a188-a654-4c2f-b5bb-e2f18ad4474e
13042023-12-15T01:12:57.358ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_826ad2ba-d609-43a3-9165-9e544e735622") failed: Property not found. retry in 738.875567ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_826ad2ba-d609-43a3-9165-9e544e735622
13052023-12-15T01:12:57.371ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_8b0e0450-abc8-4227-b368-5e4185b52841") failed: Property not found. retry in 1.274019395s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_8b0e0450-abc8-4227-b368-5e4185b52841
13062023-12-15T01:12:57.387ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_7198c8dc-786b-4865-a05e-06e4223227c4") failed: Property not found. retry in 1.090464129s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_7198c8dc-786b-4865-a05e-06e4223227c4
13072023-12-15T01:12:57.539ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_c710b263-6e30-4f78-bb21-7a9e382a9c52") failed: Property not found. retry in 667.668331ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_c710b263-6e30-4f78-bb21-7a9e382a9c52
13082023-12-15T01:12:57.648ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_df48995f-8d48-4496-b8a4-000ccf1c6fe7") failed: Property not found. retry in 1.092150392s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_df48995f-8d48-4496-b8a4-000ccf1c6fe7
13092023-12-15T01:12:57.691ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_04219130-934c-4126-868a-697ce1f64c38") failed: Property not found. retry in 723.192813ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_04219130-934c-4126-868a-697ce1f64c38
13102023-12-15T01:12:57.714ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_2217a76b-b58b-4ada-a4c4-e84eca8d38a4") failed: Property not found. retry in 930.522847ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_2217a76b-b58b-4ada-a4c4-e84eca8d38a4
13112023-12-15T01:12:57.725ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_cd0082d8-88ea-4123-9209-754395e437e9") failed: Property not found. retry in 622.675404ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_cd0082d8-88ea-4123-9209-754395e437e9
13122023-12-15T01:12:57.872ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_88b23891-1726-4275-973f-1bfd9c7d86da") failed: Property not found. retry in 1.423310008s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_88b23891-1726-4275-973f-1bfd9c7d86da
13132023-12-15T01:12:57.888ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_23e76ebf-1083-415c-b1cc-78b70ea0ee28") failed: Property not found. retry in 981.954505ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_23e76ebf-1083-415c-b1cc-78b70ea0ee28
13142023-12-15T01:12:57.969ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_3fd5ef26-c842-4d29-8079-afd0b75a8502") failed: Property not found. retry in 576.68548ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_3fd5ef26-c842-4d29-8079-afd0b75a8502
13152023-12-15T01:12:58.085ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_e9d9360e-0097-44f3-aee0-39dd0bc57e9b") failed: Property not found. retry in 980.454333ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_e9d9360e-0097-44f3-aee0-39dd0bc57e9b
13162023-12-15T01:12:58.106ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_826ad2ba-d609-43a3-9165-9e544e735622") failed: Property not found. retry in 1.098347442s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_826ad2ba-d609-43a3-9165-9e544e735622
13172023-12-15T01:12:58.129ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_127efcba-5993-4577-bcb2-f4a98a4520bb") failed: Property not found. retry in 666.280994ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_127efcba-5993-4577-bcb2-f4a98a4520bb
13182023-12-15T01:12:58.208ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_326e1e2f-55db-4ad4-aa67-d2bb64a872b8") failed: Property not found. retry in 903.408635ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_326e1e2f-55db-4ad4-aa67-d2bb64a872b8
13192023-12-15T01:12:58.216ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_c710b263-6e30-4f78-bb21-7a9e382a9c52") failed: Property not found. retry in 647.186701ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_c710b263-6e30-4f78-bb21-7a9e382a9c52
13202023-12-15T01:12:58.229ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_72d8f37c-a72e-4800-9c47-39f6b4e0d24b") failed: Property not found. retry in 1.003591475s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_72d8f37c-a72e-4800-9c47-39f6b4e0d24b
13212023-12-15T01:12:58.240ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_1756a188-a654-4c2f-b5bb-e2f18ad4474e") failed: Property not found. retry in 1.02361572s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_1756a188-a654-4c2f-b5bb-e2f18ad4474e
13222023-12-15T01:12:58.354ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_cd0082d8-88ea-4123-9209-754395e437e9") failed: Property not found. retry in 761.040573ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_cd0082d8-88ea-4123-9209-754395e437e9
13232023-12-15T01:12:58.387ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_700e7b49-f840-40d5-92e8-83ab758ea8a7") failed: Property not found. retry in 697.171409ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_700e7b49-f840-40d5-92e8-83ab758ea8a7
13242023-12-15T01:12:58.424ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_04219130-934c-4126-868a-697ce1f64c38") failed: Property not found. retry in 813.381492ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_04219130-934c-4126-868a-697ce1f64c38
13252023-12-15T01:12:58.461ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_17ef7150-35b2-46ab-918a-0b0af0a9b49d") failed: Property not found. retry in 695.749728ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_17ef7150-35b2-46ab-918a-0b0af0a9b49d
13262023-12-15T01:12:58.486ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_7198c8dc-786b-4865-a05e-06e4223227c4") failed: Property not found. retry in 1.251536622s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_7198c8dc-786b-4865-a05e-06e4223227c4
13272023-12-15T01:12:58.553ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_3fd5ef26-c842-4d29-8079-afd0b75a8502") failed: Property not found. retry in 1.271176317s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_3fd5ef26-c842-4d29-8079-afd0b75a8502
13282023-12-15T01:12:58.564ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_5b1f3c9f-bb97-4778-92ff-b6ab4fa6d3f5") failed: Property not found. retry in 1.090806867s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_5b1f3c9f-bb97-4778-92ff-b6ab4fa6d3f5
13292023-12-15T01:12:58.651ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_8b0e0450-abc8-4227-b368-5e4185b52841") failed: Property not found. retry in 910.712045ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_8b0e0450-abc8-4227-b368-5e4185b52841
13302023-12-15T01:12:58.659ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_2217a76b-b58b-4ada-a4c4-e84eca8d38a4") failed: Property not found. retry in 573.670724ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_2217a76b-b58b-4ada-a4c4-e84eca8d38a4
13312023-12-15T01:12:58.749ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_df48995f-8d48-4496-b8a4-000ccf1c6fe7") failed: Property not found. retry in 883.95918ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_df48995f-8d48-4496-b8a4-000ccf1c6fe7
13322023-12-15T01:12:58.849ZINFOSledAgent (ServiceManager): Ensuring address fd00:1122:3344:101::d exists file = sled-agent/src/services.rs:1764
13332023-12-15T01:12:58.850ZINFOSledAgent (ServiceManager): Adding address: Static(V6(Ipv6Network { addr: fd00:1122:3344:101::d, prefix: 64 })) file = illumos-utils/src/running_zone.rs:608 zone = oxz_oximeter_127efcba-5993-4577-bcb2-f4a98a4520bb
13342023-12-15T01:12:59.415ZINFOSledAgent (ServiceManager): Ensuring address fd00:1122:3344:101::d exists - OK file = sled-agent/src/services.rs:1772
13352023-12-15T01:12:59.415ZINFOSledAgent (ServiceManager): Zone using sled underlay as gateway file = sled-agent/src/services.rs:1791
13362023-12-15T01:13:14.411ZINFOSledAgent (ServiceManager): Setting up oximeter service file = sled-agent/src/services.rs:2059
13372023-12-15T01:13:14.749ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:769 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:52295
13382023-12-15T01:13:14.755ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::605a:15ff:fea0:c066": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
13392023-12-15T01:13:14.755ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 680 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:52295 req_id = d7c1b859-96aa-4ce2-a29f-9cbb97e55e75 response_code = 200 uri = /baseboard
13402023-12-15T01:13:15.022ZINFOSledAgent (ServiceManager): Ensuring address fd00:1122:3344:101::8 exists file = sled-agent/src/services.rs:1764
13412023-12-15T01:13:15.022ZINFOSledAgent (ServiceManager): Adding address: Static(V6(Ipv6Network { addr: fd00:1122:3344:101::8, prefix: 64 })) file = illumos-utils/src/running_zone.rs:608 zone = oxz_external_dns_5b1f3c9f-bb97-4778-92ff-b6ab4fa6d3f5
13422023-12-15T01:13:15.577ZINFOSledAgent (ServiceManager): Ensuring address fd00:1122:3344:101::8 exists - OK file = sled-agent/src/services.rs:1772
13432023-12-15T01:13:15.577ZINFOSledAgent (ServiceManager): Zone using sled underlay as gateway file = sled-agent/src/services.rs:1791
13442023-12-15T01:13:16.711ZINFOSledAgent (ServiceManager): Setting up external-dns service file = sled-agent/src/services.rs:1948
13452023-12-15T01:13:16.711ZINFOSledAgent (ServiceManager): Ensuring address for OPTE port file = illumos-utils/src/running_zone.rs:652 zone = oxz_external_dns_5b1f3c9f-bb97-4778-92ff-b6ab4fa6d3f5
13462023-12-15T01:13:19.757ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::605a:15ff:fea0:c066": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
13472023-12-15T01:13:22.978ZINFOSledAgent (ServiceManager): Ensuring address fd00:1122:3344:101::b exists file = sled-agent/src/services.rs:1764
13482023-12-15T01:13:22.978ZINFOSledAgent (ServiceManager): Adding address: Static(V6(Ipv6Network { addr: fd00:1122:3344:101::b, prefix: 64 })) file = illumos-utils/src/running_zone.rs:608 zone = oxz_nexus_7198c8dc-786b-4865-a05e-06e4223227c4
13492023-12-15T01:13:23.541ZINFOSledAgent (ServiceManager): Ensuring address fd00:1122:3344:101::b exists - OK file = sled-agent/src/services.rs:1772
13502023-12-15T01:13:23.541ZINFOSledAgent (ServiceManager): Zone using sled underlay as gateway file = sled-agent/src/services.rs:1791
13512023-12-15T01:13:23.972ZINFOSledAgent (ServiceManager): Setting up Nexus service file = sled-agent/src/services.rs:1835
13522023-12-15T01:13:23.972ZINFOSledAgent (ServiceManager): Ensuring address for OPTE port file = illumos-utils/src/running_zone.rs:652 zone = oxz_nexus_7198c8dc-786b-4865-a05e-06e4223227c4
13532023-12-15T01:13:24.761ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::605a:15ff:fea0:c066": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
13542023-12-15T01:13:29.763ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::605a:15ff:fea0:c066": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
13552023-12-15T01:13:29.923ZINFOSledAgent (ServiceManager): Ensuring address fd00:1122:3344:101::a exists file = sled-agent/src/services.rs:1764
13562023-12-15T01:13:29.923ZINFOSledAgent (ServiceManager): Adding address: Static(V6(Ipv6Network { addr: fd00:1122:3344:101::a, prefix: 64 })) file = illumos-utils/src/running_zone.rs:608 zone = oxz_nexus_72d8f37c-a72e-4800-9c47-39f6b4e0d24b
13572023-12-15T01:13:30.477ZINFOSledAgent (ServiceManager): Ensuring address fd00:1122:3344:101::a exists - OK file = sled-agent/src/services.rs:1772
13582023-12-15T01:13:30.477ZINFOSledAgent (ServiceManager): Zone using sled underlay as gateway file = sled-agent/src/services.rs:1791
13592023-12-15T01:13:30.480ZWARNSledAgent (StorageMonitor): failed to notify nexus about a new pool oxp_cd70d7f6-2354-4bf2-8012-55bf9eaf7930 on disk DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-cd70d7f6-2354-4bf2-8012-55bf9eaf7930", model: "synthetic-model" } err = "Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/sled-agents/78829061-d92a-47ac-bcd6-186a0b9f0582/zpools/cd70d7f6-2354-4bf2-8012-55bf9eaf7930): error trying to connect: tcp connect error: Connection refused (os error 146)" file = sled-agent/src/storage_monitor.rs:273 total duration = 269.969254157s
13602023-12-15T01:13:30.884ZINFOSledAgent (ServiceManager): Setting up Nexus service file = sled-agent/src/services.rs:1835
13612023-12-15T01:13:30.884ZINFOSledAgent (ServiceManager): Ensuring address for OPTE port file = illumos-utils/src/running_zone.rs:652 zone = oxz_nexus_72d8f37c-a72e-4800-9c47-39f6b4e0d24b
13622023-12-15T01:13:31.440ZWARNSledAgent (StorageMonitor): failed to notify nexus about a new pool oxp_616b26df-e62a-4c68-b506-f4a923d8aaf7 on disk DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-616b26df-e62a-4c68-b506-f4a923d8aaf7", model: "synthetic-model" } err = "Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/sled-agents/78829061-d92a-47ac-bcd6-186a0b9f0582/zpools/616b26df-e62a-4c68-b506-f4a923d8aaf7): error trying to connect: tcp connect error: Connection refused (os error 146)" file = sled-agent/src/storage_monitor.rs:273 total duration = 270.929382725s
13632023-12-15T01:13:32.060ZWARNSledAgent (StorageMonitor): failed to notify nexus about a new pool oxp_e4b4dc87-ab46-49fb-a4b4-d361ae214c03 on disk DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-e4b4dc87-ab46-49fb-a4b4-d361ae214c03", model: "synthetic-model" } err = "Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/sled-agents/78829061-d92a-47ac-bcd6-186a0b9f0582/zpools/e4b4dc87-ab46-49fb-a4b4-d361ae214c03): error trying to connect: tcp connect error: Connection refused (os error 146)" file = sled-agent/src/storage_monitor.rs:273 total duration = 271.548844893s
13642023-12-15T01:13:32.770ZWARNSledAgent: failed to register as a metric producer with Nexus error = "Metric registration error: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/metrics/producers): error trying to connect: tcp connect error: Connection refused (os error 146)" file = sled-agent/src/sled_agent.rs:1076 retry_after = 13.372824914s sled_id = 78829061-d92a-47ac-bcd6-186a0b9f0582
13652023-12-15T01:13:33.040ZWARNSledAgent (StorageMonitor): failed to notify nexus about a new pool oxp_14b4dc87-ab46-49fb-a4b4-d361ae214c03 on disk DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-14b4dc87-ab46-49fb-a4b4-d361ae214c03", model: "synthetic-model" } err = "Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/sled-agents/78829061-d92a-47ac-bcd6-186a0b9f0582/zpools/14b4dc87-ab46-49fb-a4b4-d361ae214c03): error trying to connect: tcp connect error: Connection refused (os error 146)" file = sled-agent/src/storage_monitor.rs:273 total duration = 272.530129701s
13662023-12-15T01:13:33.040ZWARNSledAgent (StorageMonitor): failed to notify nexus about a new pool oxp_24b4dc87-ab46-49fb-a4b4-d361ae214c03 on disk DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-24b4dc87-ab46-49fb-a4b4-d361ae214c03", model: "synthetic-model" } err = "Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/sled-agents/78829061-d92a-47ac-bcd6-186a0b9f0582/zpools/24b4dc87-ab46-49fb-a4b4-d361ae214c03): error trying to connect: tcp connect error: Connection refused (os error 146)" file = sled-agent/src/storage_monitor.rs:273 total duration = 272.53003428s
13672023-12-15T01:13:33.041ZWARNSledAgent (StorageMonitor): failed to notify nexus about a new pool oxi_a462a7f7-b628-40fe-80ff-4e4189e2d62b on disk DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-a462a7f7-b628-40fe-80ff-4e4189e2d62b", model: "synthetic-model" } err = "Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/sled-agents/78829061-d92a-47ac-bcd6-186a0b9f0582/zpools/a462a7f7-b628-40fe-80ff-4e4189e2d62b): error trying to connect: tcp connect error: Connection refused (os error 146)" file = sled-agent/src/storage_monitor.rs:273 total duration = 272.529971699s
13682023-12-15T01:13:33.041ZWARNSledAgent (StorageMonitor): failed to notify nexus about Put(PhysicalDiskPutRequest { model: "synthetic-model", serial: "synthetic-serial-cd70d7f6-2354-4bf2-8012-55bf9eaf7930", sled_id: 78829061-d92a-47ac-bcd6-186a0b9f0582, variant: U2, vendor: "synthetic-vendor" }) err = "Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/physical-disk): error trying to connect: tcp connect error: Connection refused (os error 146)" file = sled-agent/src/storage_monitor.rs:223 total duration = 272.531462481s
13692023-12-15T01:13:33.041ZWARNSledAgent (StorageMonitor): failed to notify nexus about Put(PhysicalDiskPutRequest { model: "synthetic-model", serial: "synthetic-serial-14b4dc87-ab46-49fb-a4b4-d361ae214c03", sled_id: 78829061-d92a-47ac-bcd6-186a0b9f0582, variant: U2, vendor: "synthetic-vendor" }) err = "Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/physical-disk): error trying to connect: tcp connect error: Connection refused (os error 146)" file = sled-agent/src/storage_monitor.rs:223 total duration = 272.533149246s
13702023-12-15T01:13:33.041ZWARNSledAgent (StorageMonitor): failed to notify nexus about a new pool oxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03 on disk DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-f4b4dc87-ab46-49fb-a4b4-d361ae214c03", model: "synthetic-model" } err = "Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/sled-agents/78829061-d92a-47ac-bcd6-186a0b9f0582/zpools/f4b4dc87-ab46-49fb-a4b4-d361ae214c03): error trying to connect: tcp connect error: Connection refused (os error 146)" file = sled-agent/src/storage_monitor.rs:273 total duration = 272.52939317s
13712023-12-15T01:13:33.400ZWARNSledAgent (StorageMonitor): failed to notify nexus about Put(PhysicalDiskPutRequest { model: "synthetic-model", serial: "synthetic-serial-24b4dc87-ab46-49fb-a4b4-d361ae214c03", sled_id: 78829061-d92a-47ac-bcd6-186a0b9f0582, variant: U2, vendor: "synthetic-vendor" }) err = "Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/physical-disk): error trying to connect: tcp connect error: Connection refused (os error 146)" file = sled-agent/src/storage_monitor.rs:223 total duration = 272.891575584s
13722023-12-15T01:13:33.960ZWARNSledAgent (StorageMonitor): failed to notify nexus about a new pool oxi_b462a7f7-b628-40fe-80ff-4e4189e2d62b on disk DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-b462a7f7-b628-40fe-80ff-4e4189e2d62b", model: "synthetic-model" } err = "Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/sled-agents/78829061-d92a-47ac-bcd6-186a0b9f0582/zpools/b462a7f7-b628-40fe-80ff-4e4189e2d62b): error trying to connect: tcp connect error: Connection refused (os error 146)" file = sled-agent/src/storage_monitor.rs:273 total duration = 273.448840715s
13732023-12-15T01:13:34.766ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::605a:15ff:fea0:c066": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
13742023-12-15T01:13:36.046ZINFOSledAgent (ServiceManager): Ensuring address fd00:1122:3344:101::9 exists file = sled-agent/src/services.rs:1764
13752023-12-15T01:13:36.046ZINFOSledAgent (ServiceManager): Adding address: Static(V6(Ipv6Network { addr: fd00:1122:3344:101::9, prefix: 64 })) file = illumos-utils/src/running_zone.rs:608 zone = oxz_external_dns_326e1e2f-55db-4ad4-aa67-d2bb64a872b8
13762023-12-15T01:13:36.624ZINFOSledAgent (ServiceManager): Ensuring address fd00:1122:3344:101::9 exists - OK file = sled-agent/src/services.rs:1772
13772023-12-15T01:13:36.625ZINFOSledAgent (ServiceManager): Zone using sled underlay as gateway file = sled-agent/src/services.rs:1791
13782023-12-15T01:13:36.872ZINFOSledAgent (ServiceManager): Setting up external-dns service file = sled-agent/src/services.rs:1948
13792023-12-15T01:13:36.872ZINFOSledAgent (ServiceManager): Ensuring address for OPTE port file = illumos-utils/src/running_zone.rs:652 zone = oxz_external_dns_326e1e2f-55db-4ad4-aa67-d2bb64a872b8
13802023-12-15T01:13:39.767ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::605a:15ff:fea0:c066": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
13812023-12-15T01:13:41.840ZINFOSledAgent (ServiceManager): Ensuring address fd00:1122:3344:101::c exists file = sled-agent/src/services.rs:1764
13822023-12-15T01:13:41.840ZINFOSledAgent (ServiceManager): Adding address: Static(V6(Ipv6Network { addr: fd00:1122:3344:101::c, prefix: 64 })) file = illumos-utils/src/running_zone.rs:608 zone = oxz_nexus_700e7b49-f840-40d5-92e8-83ab758ea8a7
13832023-12-15T01:13:42.394ZWARNSledAgent (StorageMonitor): failed to notify nexus about a new pool oxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03 on disk DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-f4b4dc87-ab46-49fb-a4b4-d361ae214c03", model: "synthetic-model" } err = "Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/sled-agents/78829061-d92a-47ac-bcd6-186a0b9f0582/zpools/f4b4dc87-ab46-49fb-a4b4-d361ae214c03): error trying to connect: tcp connect error: Connection refused (os error 146)" file = sled-agent/src/storage_monitor.rs:273 total duration = 281.882547129s
13842023-12-15T01:13:42.399ZINFOSledAgent (ServiceManager): Ensuring address fd00:1122:3344:101::c exists - OK file = sled-agent/src/services.rs:1772
13852023-12-15T01:13:42.399ZINFOSledAgent (ServiceManager): Zone using sled underlay as gateway file = sled-agent/src/services.rs:1791
13862023-12-15T01:13:42.678ZINFOSledAgent (ServiceManager): Setting up Nexus service file = sled-agent/src/services.rs:1835
13872023-12-15T01:13:42.678ZINFOSledAgent (ServiceManager): Ensuring address for OPTE port file = illumos-utils/src/running_zone.rs:652 zone = oxz_nexus_700e7b49-f840-40d5-92e8-83ab758ea8a7
13882023-12-15T01:13:42.910ZWARNSledAgent (StorageMonitor): failed to notify nexus about a new pool oxp_616b26df-e62a-4c68-b506-f4a923d8aaf7 on disk DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-616b26df-e62a-4c68-b506-f4a923d8aaf7", model: "synthetic-model" } err = "Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/sled-agents/78829061-d92a-47ac-bcd6-186a0b9f0582/zpools/616b26df-e62a-4c68-b506-f4a923d8aaf7): error trying to connect: tcp connect error: Connection refused (os error 146)" file = sled-agent/src/storage_monitor.rs:273 total duration = 282.399570744s
13892023-12-15T01:13:43.690ZWARNSledAgent (StorageMonitor): failed to notify nexus about a new pool oxp_e4b4dc87-ab46-49fb-a4b4-d361ae214c03 on disk DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-e4b4dc87-ab46-49fb-a4b4-d361ae214c03", model: "synthetic-model" } err = "Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/sled-agents/78829061-d92a-47ac-bcd6-186a0b9f0582/zpools/e4b4dc87-ab46-49fb-a4b4-d361ae214c03): error trying to connect: tcp connect error: Connection refused (os error 146)" file = sled-agent/src/storage_monitor.rs:273 total duration = 283.178608876s
13902023-12-15T01:13:43.790ZWARNSledAgent (StorageMonitor): failed to notify nexus about a new pool oxp_14b4dc87-ab46-49fb-a4b4-d361ae214c03 on disk DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-14b4dc87-ab46-49fb-a4b4-d361ae214c03", model: "synthetic-model" } err = "Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/sled-agents/78829061-d92a-47ac-bcd6-186a0b9f0582/zpools/14b4dc87-ab46-49fb-a4b4-d361ae214c03): error trying to connect: tcp connect error: Connection refused (os error 146)" file = sled-agent/src/storage_monitor.rs:273 total duration = 283.279715559s
13912023-12-15T01:13:44.769ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::605a:15ff:fea0:c066": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
13922023-12-15T01:13:44.791ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:769 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:32841
13932023-12-15T01:13:44.791ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 47 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:32841 req_id = dd4a3a8f-5339-427c-afb0-2e7257a63aa5 response_code = 200 uri = /baseboard
13942023-12-15T01:13:45.050ZWARNSledAgent (StorageMonitor): failed to notify nexus about a new pool oxp_cd70d7f6-2354-4bf2-8012-55bf9eaf7930 on disk DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-cd70d7f6-2354-4bf2-8012-55bf9eaf7930", model: "synthetic-model" } err = "Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/sled-agents/78829061-d92a-47ac-bcd6-186a0b9f0582/zpools/cd70d7f6-2354-4bf2-8012-55bf9eaf7930): error trying to connect: tcp connect error: Connection refused (os error 146)" file = sled-agent/src/storage_monitor.rs:273 total duration = 284.538755708s
13952023-12-15T01:13:45.140ZWARNSledAgent (StorageMonitor): failed to notify nexus about a new pool oxi_b462a7f7-b628-40fe-80ff-4e4189e2d62b on disk DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-b462a7f7-b628-40fe-80ff-4e4189e2d62b", model: "synthetic-model" } err = "Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/sled-agents/78829061-d92a-47ac-bcd6-186a0b9f0582/zpools/b462a7f7-b628-40fe-80ff-4e4189e2d62b): error trying to connect: tcp connect error: Connection refused (os error 146)" file = sled-agent/src/storage_monitor.rs:273 total duration = 284.628966487s
13962023-12-15T01:13:45.730ZWARNSledAgent (StorageMonitor): failed to notify nexus about a new pool oxp_24b4dc87-ab46-49fb-a4b4-d361ae214c03 on disk DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-24b4dc87-ab46-49fb-a4b4-d361ae214c03", model: "synthetic-model" } err = "Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/sled-agents/78829061-d92a-47ac-bcd6-186a0b9f0582/zpools/24b4dc87-ab46-49fb-a4b4-d361ae214c03): error trying to connect: tcp connect error: Connection refused (os error 146)" file = sled-agent/src/storage_monitor.rs:273 total duration = 285.219690384s
13972023-12-15T01:13:45.780ZWARNSledAgent (StorageMonitor): failed to notify nexus about Put(PhysicalDiskPutRequest { model: "synthetic-model", serial: "synthetic-serial-14b4dc87-ab46-49fb-a4b4-d361ae214c03", sled_id: 78829061-d92a-47ac-bcd6-186a0b9f0582, variant: U2, vendor: "synthetic-vendor" }) err = "Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/physical-disk): error trying to connect: tcp connect error: Connection refused (os error 146)" file = sled-agent/src/storage_monitor.rs:223 total duration = 285.272167613s
13982023-12-15T01:13:45.860ZWARNSledAgent (StorageMonitor): failed to notify nexus about Put(PhysicalDiskPutRequest { model: "synthetic-model", serial: "synthetic-serial-24b4dc87-ab46-49fb-a4b4-d361ae214c03", sled_id: 78829061-d92a-47ac-bcd6-186a0b9f0582, variant: U2, vendor: "synthetic-vendor" }) err = "Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/physical-disk): error trying to connect: tcp connect error: Connection refused (os error 146)" file = sled-agent/src/storage_monitor.rs:223 total duration = 285.351694671s
13992023-12-15T01:13:46.147ZWARNSledAgent: failed to register as a metric producer with Nexus error = "Metric registration error: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/metrics/producers): error trying to connect: tcp connect error: Connection refused (os error 146)" file = sled-agent/src/sled_agent.rs:1076 retry_after = 41.738692383s sled_id = 78829061-d92a-47ac-bcd6-186a0b9f0582
14002023-12-15T01:13:46.210ZWARNSledAgent (StorageMonitor): failed to notify nexus about Put(PhysicalDiskPutRequest { model: "synthetic-model", serial: "synthetic-serial-cd70d7f6-2354-4bf2-8012-55bf9eaf7930", sled_id: 78829061-d92a-47ac-bcd6-186a0b9f0582, variant: U2, vendor: "synthetic-vendor" }) err = "Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/physical-disk): error trying to connect: tcp connect error: Connection refused (os error 146)" file = sled-agent/src/storage_monitor.rs:223 total duration = 285.700513144s
14012023-12-15T01:13:46.370ZWARNSledAgent (StorageMonitor): failed to notify nexus about a new pool oxi_a462a7f7-b628-40fe-80ff-4e4189e2d62b on disk DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-a462a7f7-b628-40fe-80ff-4e4189e2d62b", model: "synthetic-model" } err = "Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/sled-agents/78829061-d92a-47ac-bcd6-186a0b9f0582/zpools/a462a7f7-b628-40fe-80ff-4e4189e2d62b): error trying to connect: tcp connect error: Connection refused (os error 146)" file = sled-agent/src/storage_monitor.rs:273 total duration = 285.858984031s
14022023-12-15T01:13:47.210ZWARNSledAgent (StorageMonitor): failed to notify nexus about a new pool oxp_e4b4dc87-ab46-49fb-a4b4-d361ae214c03 on disk DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-e4b4dc87-ab46-49fb-a4b4-d361ae214c03", model: "synthetic-model" } err = "Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/sled-agents/78829061-d92a-47ac-bcd6-186a0b9f0582/zpools/e4b4dc87-ab46-49fb-a4b4-d361ae214c03): error trying to connect: tcp connect error: Connection refused (os error 146)" file = sled-agent/src/storage_monitor.rs:273 total duration = 286.69902702s
14032023-12-15T01:13:48.023ZWARNSledAgent (StorageMonitor): failed to notify nexus about a new pool oxp_cd70d7f6-2354-4bf2-8012-55bf9eaf7930 on disk DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-cd70d7f6-2354-4bf2-8012-55bf9eaf7930", model: "synthetic-model" } err = "Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/sled-agents/78829061-d92a-47ac-bcd6-186a0b9f0582/zpools/cd70d7f6-2354-4bf2-8012-55bf9eaf7930): error trying to connect: tcp connect error: Connection refused (os error 146)" file = sled-agent/src/storage_monitor.rs:273 total duration = 287.511901021s
14042023-12-15T01:13:48.734ZWARNSledAgent (StorageMonitor): failed to notify nexus about a new pool oxi_b462a7f7-b628-40fe-80ff-4e4189e2d62b on disk DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-b462a7f7-b628-40fe-80ff-4e4189e2d62b", model: "synthetic-model" } err = "Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/sled-agents/78829061-d92a-47ac-bcd6-186a0b9f0582/zpools/b462a7f7-b628-40fe-80ff-4e4189e2d62b): error trying to connect: tcp connect error: Connection refused (os error 146)" file = sled-agent/src/storage_monitor.rs:273 total duration = 288.222972999s
14052023-12-15T01:13:48.890ZWARNSledAgent (StorageMonitor): failed to notify nexus about a new pool oxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03 on disk DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-f4b4dc87-ab46-49fb-a4b4-d361ae214c03", model: "synthetic-model" } err = "Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/sled-agents/78829061-d92a-47ac-bcd6-186a0b9f0582/zpools/f4b4dc87-ab46-49fb-a4b4-d361ae214c03): error trying to connect: tcp connect error: Connection refused (os error 146)" file = sled-agent/src/storage_monitor.rs:273 total duration = 288.378099475s
14062023-12-15T01:13:49.289ZINFOSledAgent (ServiceManager): Writing ledger to /pool/int/a462a7f7-b628-40fe-80ff-4e4189e2d62b/config/.omicron-zones.json.tmp file = common/src/ledger.rs:198
14072023-12-15T01:13:49.291ZINFOSledAgent (ServiceManager): Writing ledger to /pool/int/b462a7f7-b628-40fe-80ff-4e4189e2d62b/config/.omicron-zones.json.tmp file = common/src/ledger.rs:198
14082023-12-15T01:13:49.292ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 74759572 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::1]:41531 req_id = 40dab884-0af9-4ff1-b052-73c39937884b response_code = 204 uri = /omicron-zones
14092023-12-15T01:13:49.297ZINFOSledAgent (RSS): Finished setting up services file = sled-agent/src/rack_setup/service.rs:1062
14102023-12-15T01:13:49.297ZINFOSledAgent (RSS): Writing ledger to /pool/int/a462a7f7-b628-40fe-80ff-4e4189e2d62b/config/.rss-plan-completed.marker.tmp file = common/src/ledger.rs:198
14112023-12-15T01:13:49.297ZINFOSledAgent (RSS): Writing ledger to /pool/int/b462a7f7-b628-40fe-80ff-4e4189e2d62b/config/.rss-plan-completed.marker.tmp file = common/src/ledger.rs:198
14122023-12-15T01:13:49.299ZINFOSledAgent (RSS): Handing off control to Nexus file = sled-agent/src/rack_setup/service.rs:549
14132023-12-15T01:13:49.299ZINFOSledAgent (RSS): Nexus address: [fd00:1122:3344:101::c]:12221 file = sled-agent/src/rack_setup/service.rs:551
14142023-12-15T01:13:49.331ZINFOSledAgent (RSS): rack_network_config: Some( RackNetworkConfigV1 { bgp: [], infra_ip_first: 10.151.2.174, infra_ip_last: 10.151.2.174, ports: [ PortConfigV1 { addresses: [ V4( Ipv4Network { addr: 10.151.2.174, prefix: 32, }, ), ], autoneg: false, bgp_peers: [], port: "qsfp0", routes: [ RouteConfig { destination: V4( Ipv4Network { addr: 0.0.0.0, prefix: 0, }, ), nexthop: 10.151.2.100, }, ], switch: Switch0, uplink_port_fec: None, uplink_port_speed: Speed40G, }, ], rack_subnet: Ipv6Network { addr: fd00:1122:3344:1::, prefix: 56, }, }, ) file = sled-agent/src/rack_setup/service.rs:658
14152023-12-15T01:13:49.333ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/racks/5628bca7-ce21-4723-b380-345b839f80b4/initialization-complete): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/rack_setup/service.rs:679
14162023-12-15T01:13:49.442ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/racks/5628bca7-ce21-4723-b380-345b839f80b4/initialization-complete): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/rack_setup/service.rs:679
14172023-12-15T01:13:49.508ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/racks/5628bca7-ce21-4723-b380-345b839f80b4/initialization-complete): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/rack_setup/service.rs:679
14182023-12-15T01:13:49.637ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/racks/5628bca7-ce21-4723-b380-345b839f80b4/initialization-complete): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/rack_setup/service.rs:679
14192023-12-15T01:13:49.771ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::605a:15ff:fea0:c066": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
14202023-12-15T01:13:49.888ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/racks/5628bca7-ce21-4723-b380-345b839f80b4/initialization-complete): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/rack_setup/service.rs:679
14212023-12-15T01:13:51.138ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "16d756b0-81c6-42eb-8a8f-6f58d7caf1b5", "content-length": "124", "date": "Fri, 15 Dec 2023 01:13:51 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "16d756b0-81c6-42eb-8a8f-6f58d7caf1b5" } file = sled-agent/src/rack_setup/service.rs:679
14222023-12-15T01:13:51.359ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "e438d74e-a8e6-41b1-b1eb-3f60300da9a8", "content-length": "124", "date": "Fri, 15 Dec 2023 01:13:51 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "e438d74e-a8e6-41b1-b1eb-3f60300da9a8" } file = sled-agent/src/rack_setup/service.rs:679
14232023-12-15T01:13:51.378ZWARNSledAgent (StorageMonitor): failed to notify nexus about Put(PhysicalDiskPutRequest { model: "synthetic-model", serial: "synthetic-serial-24b4dc87-ab46-49fb-a4b4-d361ae214c03", sled_id: 78829061-d92a-47ac-bcd6-186a0b9f0582, variant: U2, vendor: "synthetic-vendor" }) err = "Error Response: status: 404 Not Found; headers: {\\"content-type\\": \\"application/json\\", \\"x-request-id\\": \\"e8a42dbd-6ea3-4b91-bab6-0e37a3021fc4\\", \\"content-length\\": \\"173\\", \\"date\\": \\"Fri, 15 Dec 2023 01:13:51 GMT\\"}; value: Error { error_code: Some(\\"ObjectNotFound\\"), message: \\"not found: sled with id \\\\\\"78829061-d92a-47ac-bcd6-186a0b9f0582\\\\\\"\\", request_id: \\"e8a42dbd-6ea3-4b91-bab6-0e37a3021fc4\\" }" file = sled-agent/src/storage_monitor.rs:223 total duration = 290.869331222s
14242023-12-15T01:13:51.433ZWARNSledAgent (StorageMonitor): failed to notify nexus about Put(PhysicalDiskPutRequest { model: "synthetic-model", serial: "synthetic-serial-cd70d7f6-2354-4bf2-8012-55bf9eaf7930", sled_id: 78829061-d92a-47ac-bcd6-186a0b9f0582, variant: U2, vendor: "synthetic-vendor" }) err = "Error Response: status: 404 Not Found; headers: {\\"content-type\\": \\"application/json\\", \\"x-request-id\\": \\"694830ad-5ed4-46d0-8884-5e78efb7167b\\", \\"content-length\\": \\"173\\", \\"date\\": \\"Fri, 15 Dec 2023 01:13:51 GMT\\"}; value: Error { error_code: Some(\\"ObjectNotFound\\"), message: \\"not found: sled with id \\\\\\"78829061-d92a-47ac-bcd6-186a0b9f0582\\\\\\"\\", request_id: \\"694830ad-5ed4-46d0-8884-5e78efb7167b\\" }" file = sled-agent/src/storage_monitor.rs:223 total duration = 290.923077121s
14252023-12-15T01:13:51.549ZWARNSledAgent (StorageMonitor): failed to notify nexus about a new pool oxp_31bd71cd-4736-4a12-a387-9b74b050396f on disk DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-31bd71cd-4736-4a12-a387-9b74b050396f", model: "synthetic-model" } err = "Error Response: status: 404 Not Found; headers: {\\"content-type\\": \\"application/json\\", \\"x-request-id\\": \\"0c49c81f-5aa6-4bc6-9cdf-531e362d8e3e\\", \\"content-length\\": \\"272\\", \\"date\\": \\"Fri, 15 Dec 2023 01:13:51 GMT\\"}; value: Error { error_code: Some(\\"ObjectNotFound\\"), message: \\"not found: physical-disk with id \\\\\\"vendor = \\\\\\"synthetic-vendor\\\\\\", serial = \\\\\\"synthetic-serial-31bd71cd-4736-4a12-a387-9b74b050396f\\\\\\", model = \\\\\\"synthetic-model\\\\\\"\\\\\\"\\", request_id: \\"0c49c81f-5aa6-4bc6-9cdf-531e362d8e3e\\" }" file = sled-agent/src/storage_monitor.rs:273 total duration = 291.038462208s
14262023-12-15T01:13:51.594ZINFOSledAgent (HardwareMonitor): contacting server nexus, registering sled baseboard = Baseboard { part_number: "i86pc", revision: 0, serial_number: "37fc7d027c1a4e9eb4f7b462ba03b0a1" } file = sled-agent/src/sled_agent.rs:637 id = 78829061-d92a-47ac-bcd6-186a0b9f0582
14272023-12-15T01:13:52.087ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 404 Not Found; headers: {"content-type": "application/json", "x-request-id": "0562cee0-5564-4acd-8178-591d9181d787", "content-length": "174", "date": "Fri, 15 Dec 2023 01:13:51 GMT"}; value: Error { error_code: Some("ObjectNotFound"), message: "not found: zpool with id \"14b4dc87-ab46-49fb-a4b4-d361ae214c03\"", request_id: "0562cee0-5564-4acd-8178-591d9181d787" } file = sled-agent/src/rack_setup/service.rs:679
14282023-12-15T01:13:52.282ZWARNSledAgent (StorageMonitor): failed to notify nexus about a new pool oxp_14b4dc87-ab46-49fb-a4b4-d361ae214c03 on disk DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-14b4dc87-ab46-49fb-a4b4-d361ae214c03", model: "synthetic-model" } err = "Error Response: status: 404 Not Found; headers: {\\"content-type\\": \\"application/json\\", \\"x-request-id\\": \\"3669b467-706d-49d0-ae15-71936eee7d1f\\", \\"content-length\\": \\"272\\", \\"date\\": \\"Fri, 15 Dec 2023 01:13:52 GMT\\"}; value: Error { error_code: Some(\\"ObjectNotFound\\"), message: \\"not found: physical-disk with id \\\\\\"vendor = \\\\\\"synthetic-vendor\\\\\\", serial = \\\\\\"synthetic-serial-14b4dc87-ab46-49fb-a4b4-d361ae214c03\\\\\\", model = \\\\\\"synthetic-model\\\\\\"\\\\\\"\\", request_id: \\"3669b467-706d-49d0-ae15-71936eee7d1f\\" }" file = sled-agent/src/storage_monitor.rs:273 total duration = 291.771368055s
14292023-12-15T01:13:52.903ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 404 Not Found; headers: {"content-type": "application/json", "x-request-id": "32b93ffc-3178-4f2b-8bed-372d85853b23", "content-length": "174", "date": "Fri, 15 Dec 2023 01:13:52 GMT"}; value: Error { error_code: Some("ObjectNotFound"), message: "not found: zpool with id \"14b4dc87-ab46-49fb-a4b4-d361ae214c03\"", request_id: "32b93ffc-3178-4f2b-8bed-372d85853b23" } file = sled-agent/src/rack_setup/service.rs:679
14302023-12-15T01:13:53.592ZWARNSledAgent (StorageMonitor): failed to notify nexus about a new pool oxp_24b4dc87-ab46-49fb-a4b4-d361ae214c03 on disk DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-24b4dc87-ab46-49fb-a4b4-d361ae214c03", model: "synthetic-model" } err = "Error Response: status: 404 Not Found; headers: {\\"content-type\\": \\"application/json\\", \\"x-request-id\\": \\"471a3acf-d484-4c79-8b66-34a59cc612ae\\", \\"content-length\\": \\"272\\", \\"date\\": \\"Fri, 15 Dec 2023 01:13:53 GMT\\"}; value: Error { error_code: Some(\\"ObjectNotFound\\"), message: \\"not found: physical-disk with id \\\\\\"vendor = \\\\\\"synthetic-vendor\\\\\\", serial = \\\\\\"synthetic-serial-24b4dc87-ab46-49fb-a4b4-d361ae214c03\\\\\\", model = \\\\\\"synthetic-model\\\\\\"\\\\\\"\\", request_id: \\"471a3acf-d484-4c79-8b66-34a59cc612ae\\" }" file = sled-agent/src/storage_monitor.rs:273 total duration = 293.08115138s
14312023-12-15T01:13:53.810ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 404 Not Found; headers: {"content-type": "application/json", "x-request-id": "049b604a-832c-4e79-ade0-152af427c89a", "content-length": "174", "date": "Fri, 15 Dec 2023 01:13:53 GMT"}; value: Error { error_code: Some("ObjectNotFound"), message: "not found: zpool with id \"14b4dc87-ab46-49fb-a4b4-d361ae214c03\"", request_id: "049b604a-832c-4e79-ade0-152af427c89a" } file = sled-agent/src/rack_setup/service.rs:679
14322023-12-15T01:13:54.772ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::605a:15ff:fea0:c066": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
14332023-12-15T01:13:54.921ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 404 Not Found; headers: {"content-type": "application/json", "x-request-id": "834a77bc-6a0c-4ec8-a135-d5c8effb4df4", "content-length": "174", "date": "Fri, 15 Dec 2023 01:13:54 GMT"}; value: Error { error_code: Some("ObjectNotFound"), message: "not found: zpool with id \"14b4dc87-ab46-49fb-a4b4-d361ae214c03\"", request_id: "834a77bc-6a0c-4ec8-a135-d5c8effb4df4" } file = sled-agent/src/rack_setup/service.rs:679
14342023-12-15T01:13:55.088ZINFOSledAgent (StorageMonitor): Nexus notification complete: Put(PhysicalDiskPutRequest { model: "synthetic-model", serial: "synthetic-serial-14b4dc87-ab46-49fb-a4b4-d361ae214c03", sled_id: 78829061-d92a-47ac-bcd6-186a0b9f0582, variant: U2, vendor: "synthetic-vendor" }) file = sled-agent/src/storage_monitor.rs:105
14352023-12-15T01:13:55.735ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 404 Not Found; headers: {"content-type": "application/json", "x-request-id": "400c14b6-d71e-439e-aeb2-5af7bc70f421", "content-length": "174", "date": "Fri, 15 Dec 2023 01:13:54 GMT"}; value: Error { error_code: Some("ObjectNotFound"), message: "not found: zpool with id \"14b4dc87-ab46-49fb-a4b4-d361ae214c03\"", request_id: "400c14b6-d71e-439e-aeb2-5af7bc70f421" } file = sled-agent/src/rack_setup/service.rs:679
14362023-12-15T01:13:55.747ZINFOSledAgent (StorageMonitor): Nexus notification complete: Put(PhysicalDiskPutRequest { model: "synthetic-model", serial: "synthetic-serial-24b4dc87-ab46-49fb-a4b4-d361ae214c03", sled_id: 78829061-d92a-47ac-bcd6-186a0b9f0582, variant: U2, vendor: "synthetic-vendor" }) file = sled-agent/src/storage_monitor.rs:105
14372023-12-15T01:13:55.747ZINFOSledAgent (StorageMonitor): Nexus notification complete: Put(PhysicalDiskPutRequest { model: "synthetic-model", serial: "synthetic-serial-31bd71cd-4736-4a12-a387-9b74b050396f", sled_id: 78829061-d92a-47ac-bcd6-186a0b9f0582, variant: U2, vendor: "synthetic-vendor" }) file = sled-agent/src/storage_monitor.rs:105
14382023-12-15T01:13:55.747ZINFOSledAgent (StorageMonitor): Nexus notification complete: Put(PhysicalDiskPutRequest { model: "synthetic-model", serial: "synthetic-serial-616b26df-e62a-4c68-b506-f4a923d8aaf7", sled_id: 78829061-d92a-47ac-bcd6-186a0b9f0582, variant: U2, vendor: "synthetic-vendor" }) file = sled-agent/src/storage_monitor.rs:105
14392023-12-15T01:13:55.747ZINFOSledAgent (StorageMonitor): Nexus notification complete: Put(PhysicalDiskPutRequest { model: "synthetic-model", serial: "synthetic-serial-a462a7f7-b628-40fe-80ff-4e4189e2d62b", sled_id: 78829061-d92a-47ac-bcd6-186a0b9f0582, variant: M2, vendor: "synthetic-vendor" }) file = sled-agent/src/storage_monitor.rs:105
14402023-12-15T01:13:55.747ZINFOSledAgent (StorageMonitor): Nexus notification complete: Put(PhysicalDiskPutRequest { model: "synthetic-model", serial: "synthetic-serial-b462a7f7-b628-40fe-80ff-4e4189e2d62b", sled_id: 78829061-d92a-47ac-bcd6-186a0b9f0582, variant: M2, vendor: "synthetic-vendor" }) file = sled-agent/src/storage_monitor.rs:105
14412023-12-15T01:13:56.501ZWARNSledAgent (StorageMonitor): failed to notify nexus about a new pool oxp_cd70d7f6-2354-4bf2-8012-55bf9eaf7930 on disk DiskIdentity { vendor: "synthetic-vendor", serial: "synthetic-serial-cd70d7f6-2354-4bf2-8012-55bf9eaf7930", model: "synthetic-model" } err = "Error Response: status: 404 Not Found; headers: {\\"content-type\\": \\"application/json\\", \\"x-request-id\\": \\"47f16633-3eba-40ed-a224-cb64bcd3a855\\", \\"content-length\\": \\"272\\", \\"date\\": \\"Fri, 15 Dec 2023 01:13:55 GMT\\"}; value: Error { error_code: Some(\\"ObjectNotFound\\"), message: \\"not found: physical-disk with id \\\\\\"vendor = \\\\\\"synthetic-vendor\\\\\\", serial = \\\\\\"synthetic-serial-cd70d7f6-2354-4bf2-8012-55bf9eaf7930\\\\\\", model = \\\\\\"synthetic-model\\\\\\"\\\\\\"\\", request_id: \\"47f16633-3eba-40ed-a224-cb64bcd3a855\\" }" file = sled-agent/src/storage_monitor.rs:273 total duration = 295.989607387s
14422023-12-15T01:13:56.671ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 404 Not Found; headers: {"content-type": "application/json", "x-request-id": "7a01312a-cff3-48ee-a4ac-7235b5e07f9e", "content-length": "174", "date": "Fri, 15 Dec 2023 01:13:56 GMT"}; value: Error { error_code: Some("ObjectNotFound"), message: "not found: zpool with id \"14b4dc87-ab46-49fb-a4b4-d361ae214c03\"", request_id: "7a01312a-cff3-48ee-a4ac-7235b5e07f9e" } file = sled-agent/src/rack_setup/service.rs:679
14432023-12-15T01:13:57.591ZINFOSledAgent (StorageMonitor): Nexus notification complete: Put(PhysicalDiskPutRequest { model: "synthetic-model", serial: "synthetic-serial-cd70d7f6-2354-4bf2-8012-55bf9eaf7930", sled_id: 78829061-d92a-47ac-bcd6-186a0b9f0582, variant: U2, vendor: "synthetic-vendor" }) file = sled-agent/src/storage_monitor.rs:105
14442023-12-15T01:13:57.591ZINFOSledAgent (StorageMonitor): Nexus notification complete: Put(PhysicalDiskPutRequest { model: "synthetic-model", serial: "synthetic-serial-ceb4461c-cf56-4719-ad3c-14430bfdfb60", sled_id: 78829061-d92a-47ac-bcd6-186a0b9f0582, variant: U2, vendor: "synthetic-vendor" }) file = sled-agent/src/storage_monitor.rs:105
14452023-12-15T01:13:57.591ZINFOSledAgent (StorageMonitor): Nexus notification complete: Put(PhysicalDiskPutRequest { model: "synthetic-model", serial: "synthetic-serial-d462a7f7-b628-40fe-80ff-4e4189e2d62b", sled_id: 78829061-d92a-47ac-bcd6-186a0b9f0582, variant: U2, vendor: "synthetic-vendor" }) file = sled-agent/src/storage_monitor.rs:105
14462023-12-15T01:13:57.591ZINFOSledAgent (StorageMonitor): Nexus notification complete: Put(PhysicalDiskPutRequest { model: "synthetic-model", serial: "synthetic-serial-e4b4dc87-ab46-49fb-a4b4-d361ae214c03", sled_id: 78829061-d92a-47ac-bcd6-186a0b9f0582, variant: U2, vendor: "synthetic-vendor" }) file = sled-agent/src/storage_monitor.rs:105
14472023-12-15T01:13:57.591ZINFOSledAgent (StorageMonitor): Nexus notification complete: Put(PhysicalDiskPutRequest { model: "synthetic-model", serial: "synthetic-serial-f4b4dc87-ab46-49fb-a4b4-d361ae214c03", sled_id: 78829061-d92a-47ac-bcd6-186a0b9f0582, variant: U2, vendor: "synthetic-vendor" }) file = sled-agent/src/storage_monitor.rs:105
14482023-12-15T01:13:58.354ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 404 Not Found; headers: {"content-type": "application/json", "x-request-id": "e7ad15e5-4072-4801-a80d-b552850c8da5", "content-length": "174", "date": "Fri, 15 Dec 2023 01:13:58 GMT"}; value: Error { error_code: Some("ObjectNotFound"), message: "not found: zpool with id \"14b4dc87-ab46-49fb-a4b4-d361ae214c03\"", request_id: "e7ad15e5-4072-4801-a80d-b552850c8da5" } file = sled-agent/src/rack_setup/service.rs:679
14492023-12-15T01:13:59.774ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::605a:15ff:fea0:c066": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
14502023-12-15T01:14:00.226ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 404 Not Found; headers: {"content-type": "application/json", "x-request-id": "613f41fa-4c42-4e5a-ab61-29eaaa42cc8c", "content-length": "174", "date": "Fri, 15 Dec 2023 01:13:59 GMT"}; value: Error { error_code: Some("ObjectNotFound"), message: "not found: zpool with id \"14b4dc87-ab46-49fb-a4b4-d361ae214c03\"", request_id: "613f41fa-4c42-4e5a-ab61-29eaaa42cc8c" } file = sled-agent/src/rack_setup/service.rs:679
14512023-12-15T01:14:00.340ZINFOSledAgent (StorageMonitor): Nexus notification complete: ZpoolPutRequest { disk_model: "synthetic-model", disk_serial: "synthetic-serial-14b4dc87-ab46-49fb-a4b4-d361ae214c03", disk_vendor: "synthetic-vendor", size: ByteCount(20937965568) } file = sled-agent/src/storage_monitor.rs:105
14522023-12-15T01:14:00.340ZINFOSledAgent (StorageMonitor): Nexus notification complete: ZpoolPutRequest { disk_model: "synthetic-model", disk_serial: "synthetic-serial-24b4dc87-ab46-49fb-a4b4-d361ae214c03", disk_vendor: "synthetic-vendor", size: ByteCount(20937965568) } file = sled-agent/src/storage_monitor.rs:105
14532023-12-15T01:14:00.340ZINFOSledAgent (StorageMonitor): Nexus notification complete: ZpoolPutRequest { disk_model: "synthetic-model", disk_serial: "synthetic-serial-31bd71cd-4736-4a12-a387-9b74b050396f", disk_vendor: "synthetic-vendor", size: ByteCount(20937965568) } file = sled-agent/src/storage_monitor.rs:105
14542023-12-15T01:14:00.340ZINFOSledAgent (StorageMonitor): Nexus notification complete: ZpoolPutRequest { disk_model: "synthetic-model", disk_serial: "synthetic-serial-616b26df-e62a-4c68-b506-f4a923d8aaf7", disk_vendor: "synthetic-vendor", size: ByteCount(20937965568) } file = sled-agent/src/storage_monitor.rs:105
14552023-12-15T01:14:00.340ZINFOSledAgent (StorageMonitor): Nexus notification complete: ZpoolPutRequest { disk_model: "synthetic-model", disk_serial: "synthetic-serial-a462a7f7-b628-40fe-80ff-4e4189e2d62b", disk_vendor: "synthetic-vendor", size: ByteCount(20937965568) } file = sled-agent/src/storage_monitor.rs:105
14562023-12-15T01:14:00.340ZINFOSledAgent (StorageMonitor): Nexus notification complete: ZpoolPutRequest { disk_model: "synthetic-model", disk_serial: "synthetic-serial-b462a7f7-b628-40fe-80ff-4e4189e2d62b", disk_vendor: "synthetic-vendor", size: ByteCount(20937965568) } file = sled-agent/src/storage_monitor.rs:105
14572023-12-15T01:14:01.722ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 404 Not Found; headers: {"content-type": "application/json", "x-request-id": "2c4181b3-a02b-4e7e-acc8-b1838fd483fb", "content-length": "174", "date": "Fri, 15 Dec 2023 01:14:01 GMT"}; value: Error { error_code: Some("ObjectNotFound"), message: "not found: zpool with id \"cd70d7f6-2354-4bf2-8012-55bf9eaf7930\"", request_id: "2c4181b3-a02b-4e7e-acc8-b1838fd483fb" } file = sled-agent/src/rack_setup/service.rs:679
14582023-12-15T01:14:03.755ZINFOSledAgent (StorageMonitor): Nexus notification complete: ZpoolPutRequest { disk_model: "synthetic-model", disk_serial: "synthetic-serial-cd70d7f6-2354-4bf2-8012-55bf9eaf7930", disk_vendor: "synthetic-vendor", size: ByteCount(20937965568) } file = sled-agent/src/storage_monitor.rs:105
14592023-12-15T01:14:03.755ZINFOSledAgent (StorageMonitor): Nexus notification complete: ZpoolPutRequest { disk_model: "synthetic-model", disk_serial: "synthetic-serial-ceb4461c-cf56-4719-ad3c-14430bfdfb60", disk_vendor: "synthetic-vendor", size: ByteCount(20937965568) } file = sled-agent/src/storage_monitor.rs:105
14602023-12-15T01:14:03.755ZINFOSledAgent (StorageMonitor): Nexus notification complete: ZpoolPutRequest { disk_model: "synthetic-model", disk_serial: "synthetic-serial-d462a7f7-b628-40fe-80ff-4e4189e2d62b", disk_vendor: "synthetic-vendor", size: ByteCount(20937965568) } file = sled-agent/src/storage_monitor.rs:105
14612023-12-15T01:14:03.755ZINFOSledAgent (StorageMonitor): Nexus notification complete: ZpoolPutRequest { disk_model: "synthetic-model", disk_serial: "synthetic-serial-e4b4dc87-ab46-49fb-a4b4-d361ae214c03", disk_vendor: "synthetic-vendor", size: ByteCount(20937965568) } file = sled-agent/src/storage_monitor.rs:105
14622023-12-15T01:14:03.755ZINFOSledAgent (StorageMonitor): Nexus notification complete: ZpoolPutRequest { disk_model: "synthetic-model", disk_serial: "synthetic-serial-f4b4dc87-ab46-49fb-a4b4-d361ae214c03", disk_vendor: "synthetic-vendor", size: ByteCount(20937965568) } file = sled-agent/src/storage_monitor.rs:105
14632023-12-15T01:14:04.544ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:769 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:55669
14642023-12-15T01:14:04.546ZINFOSledAgent (PortManager): Ensuring VPC firewall rules file = illumos-utils/src/opte/port_manager.rs:412 rules = [VpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: c63661fd-8921-45f4-9ea1-957f627b0035, kind: Service { id: 5b1f3c9f-bb97-4778-92ff-b6ab4fa6d3f5 }, name: Name("external-dns-5b1f3c9f-bb97-4778-92ff-b6ab4fa6d3f5"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 248, 190])), subnet: V4(Ipv4Net(Ipv4Network { addr: 172.30.1.0, prefix: 24 })), vni: Vni(100), primary: true, slot: 1 }, NetworkInterface { id: f95722db-ae4d-47db-b31f-4fe3f0644245, kind: Service { id: 326e1e2f-55db-4ad4-aa67-d2bb64a872b8 }, name: Name("external-dns-326e1e2f-55db-4ad4-aa67-d2bb64a872b8"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 155, 183])), subnet: V4(Ipv4Net(Ipv4Network { addr: 172.30.1.0, prefix: 24 })), vni: Vni(100), primary: true, slot: 1 }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, VpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 350830fe-df21-4d4e-9a2e-ab0ea31cf8b3, kind: Service { id: 700e7b49-f840-40d5-92e8-83ab758ea8a7 }, name: Name("nexus-700e7b49-f840-40d5-92e8-83ab758ea8a7"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 230, 76])), subnet: V4(Ipv4Net(Ipv4Network { addr: 172.30.2.0, prefix: 24 })), vni: Vni(100), primary: true, slot: 1 }, NetworkInterface { id: 8f13ae46-553d-48d4-9492-805134914028, kind: Service { id: 7198c8dc-786b-4865-a05e-06e4223227c4 }, name: Name("nexus-7198c8dc-786b-4865-a05e-06e4223227c4"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 227, 182])), subnet: V4(Ipv4Net(Ipv4Network { addr: 172.30.2.0, prefix: 24 })), vni: Vni(100), primary: true, slot: 1 }, NetworkInterface { id: ac0cfa1b-6ee0-4c4c-8dd0-6e3f0609fd4a, kind: Service { id: 72d8f37c-a72e-4800-9c47-39f6b4e0d24b }, name: Name("nexus-72d8f37c-a72e-4800-9c47-39f6b4e0d24b"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 160, 115])), subnet: V4(Ipv4Net(Ipv4Network { addr: 172.30.2.0, prefix: 24 })), vni: Vni(100), primary: true, slot: 1 }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }] vni = Vni(100)
14652023-12-15T01:14:04.546ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:431 port = "opte0" rules = []
14662023-12-15T01:14:04.546ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:431 port = "opte4" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
14672023-12-15T01:14:04.546ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:431 port = "opte2" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
14682023-12-15T01:14:04.546ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:431 port = "opte3" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
14692023-12-15T01:14:04.546ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:431 port = "opte1" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
14702023-12-15T01:14:04.546ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:431 port = "opte5" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
14712023-12-15T01:14:04.546ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 1443 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:55669 req_id = 79dbb339-62b5-4863-90d7-a8d97fbc305f response_code = 204 uri = /vpc/001de000-074c-4000-8000-000000000000/firewall/rules
14722023-12-15T01:14:04.775ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::605a:15ff:fea0:c066": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
14732023-12-15T01:14:04.870ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:769 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:50950
14742023-12-15T01:14:04.870ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 538 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:50950 req_id = c074f9b6-6fb4-4cc2-98ce-f9fdb03026aa response_code = 200 uri = /network-bootstore-config
14752023-12-15T01:14:04.877ZINFOSledAgent (RSS): Handoff to Nexus is complete file = sled-agent/src/rack_setup/service.rs:689
14762023-12-15T01:14:09.777ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::605a:15ff:fea0:c066": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
14772023-12-15T01:14:14.778ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::605a:15ff:fea0:c066": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
14782023-12-15T01:14:14.825ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:769 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:52632
14792023-12-15T01:14:14.825ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 36 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:52632 req_id = fc264d7c-5e4f-4dd7-95a1-1f995f7a494e response_code = 200 uri = /baseboard
14802023-12-15T01:14:19.779ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::605a:15ff:fea0:c066": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
14812023-12-15T01:14:24.781ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::605a:15ff:fea0:c066": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
14822023-12-15T01:14:29.782ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::605a:15ff:fea0:c066": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
14832023-12-15T01:14:34.783ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::605a:15ff:fea0:c066": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
14842023-12-15T01:14:39.785ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::605a:15ff:fea0:c066": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
14852023-12-15T01:14:44.252ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:532
14862023-12-15T01:14:44.786ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::605a:15ff:fea0:c066": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
14872023-12-15T01:14:44.857ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:769 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:52195
14882023-12-15T01:14:44.858ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 43 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:52195 req_id = dd918355-997e-42d2-8156-57dab55687d5 response_code = 200 uri = /baseboard
14892023-12-15T01:14:49.787ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::605a:15ff:fea0:c066": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
14902023-12-15T01:14:54.790ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::605a:15ff:fea0:c066": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
14912023-12-15T01:14:57.991ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:769 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::d]:41278
14922023-12-15T01:14:57.993ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 2115 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::d]:41278 req_id = 2d5e5f3d-0dd3-437d-badd-04b913a1cfb3 response_code = 200 uri = /metrics/collect/78829061-d92a-47ac-bcd6-186a0b9f0582
14932023-12-15T01:14:59.791ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::605a:15ff:fea0:c066": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
14942023-12-15T01:15:04.793ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::605a:15ff:fea0:c066": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
14952023-12-15T01:15:09.794ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::605a:15ff:fea0:c066": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
14962023-12-15T01:15:14.795ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::605a:15ff:fea0:c066": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
14972023-12-15T01:15:14.891ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:769 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:50675
14982023-12-15T01:15:14.891ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 32 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:50675 req_id = 8312d465-d6bb-450d-8418-795ebd0f5935 response_code = 200 uri = /baseboard
14992023-12-15T01:15:19.797ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::605a:15ff:fea0:c066": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
15002023-12-15T01:15:24.798ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::605a:15ff:fea0:c066": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
15012023-12-15T01:15:27.991ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 236 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::d]:41278 req_id = 3c6f547d-ab81-479d-8952-11ea142bf134 response_code = 200 uri = /metrics/collect/78829061-d92a-47ac-bcd6-186a0b9f0582
15022023-12-15T01:15:29.800ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::605a:15ff:fea0:c066": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
15032023-12-15T01:15:34.801ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::605a:15ff:fea0:c066": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
15042023-12-15T01:15:39.802ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::605a:15ff:fea0:c066": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
15052023-12-15T01:15:44.803ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::605a:15ff:fea0:c066": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
15062023-12-15T01:15:44.924ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:769 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:65056
15072023-12-15T01:15:44.924ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 29 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:65056 req_id = 24fed926-9f05-42e8-908c-4e6c6e10b06d response_code = 200 uri = /baseboard
15082023-12-15T01:15:49.805ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::605a:15ff:fea0:c066": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
15092023-12-15T01:15:54.806ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::605a:15ff:fea0:c066": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
15102023-12-15T01:15:57.991ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 213 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::d]:41278 req_id = 581b1db1-c250-448a-86db-d9ba42fda423 response_code = 200 uri = /metrics/collect/78829061-d92a-47ac-bcd6-186a0b9f0582
15112023-12-15T01:15:59.808ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::605a:15ff:fea0:c066": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
15122023-12-15T01:16:04.810ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::605a:15ff:fea0:c066": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
15132023-12-15T01:16:09.812ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::605a:15ff:fea0:c066": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
15142023-12-15T01:16:14.813ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::605a:15ff:fea0:c066": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
15152023-12-15T01:16:14.956ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:769 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:44040
15162023-12-15T01:16:14.956ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 32 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:44040 req_id = 0ba1a595-49f0-4abb-b4ce-46cea6305251 response_code = 200 uri = /baseboard
15172023-12-15T01:16:19.814ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::605a:15ff:fea0:c066": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
15182023-12-15T01:16:24.816ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::605a:15ff:fea0:c066": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
15192023-12-15T01:16:27.991ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 217 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::d]:41278 req_id = 510187a6-f5a2-48ac-9d63-96478cede7e4 response_code = 200 uri = /metrics/collect/78829061-d92a-47ac-bcd6-186a0b9f0582
15202023-12-15T01:16:29.818ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::605a:15ff:fea0:c066": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
15212023-12-15T01:16:34.820ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::605a:15ff:fea0:c066": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
15222023-12-15T01:16:39.822ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::605a:15ff:fea0:c066": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
15232023-12-15T01:16:44.822ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::605a:15ff:fea0:c066": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
15242023-12-15T01:16:44.988ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:769 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:54007
15252023-12-15T01:16:44.988ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 26 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:54007 req_id = 7a9d7f73-e8d2-4f7e-a4e8-fb6a030f0167 response_code = 200 uri = /baseboard
15262023-12-15T01:16:49.824ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::605a:15ff:fea0:c066": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
15272023-12-15T01:16:54.826ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::605a:15ff:fea0:c066": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
15282023-12-15T01:16:57.991ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 277 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::d]:41278 req_id = 1641cbb9-e64a-4445-885a-50978a025e53 response_code = 200 uri = /metrics/collect/78829061-d92a-47ac-bcd6-186a0b9f0582
15292023-12-15T01:16:59.828ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::605a:15ff:fea0:c066": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
15302023-12-15T01:17:04.830ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::605a:15ff:fea0:c066": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
15312023-12-15T01:17:09.830ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::605a:15ff:fea0:c066": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
15322023-12-15T01:17:14.832ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::605a:15ff:fea0:c066": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
15332023-12-15T01:17:15.021ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:769 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:61740
15342023-12-15T01:17:15.021ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 32 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:61740 req_id = fe8838e2-5624-42c2-b892-bdd83b9a7ede response_code = 200 uri = /baseboard
15352023-12-15T01:17:19.834ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::605a:15ff:fea0:c066": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
15362023-12-15T01:17:24.837ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::605a:15ff:fea0:c066": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
15372023-12-15T01:17:27.992ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 276 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::d]:41278 req_id = 577f4a3f-dc26-45d1-aae7-952aafbb4b76 response_code = 200 uri = /metrics/collect/78829061-d92a-47ac-bcd6-186a0b9f0582
15382023-12-15T01:17:29.837ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::605a:15ff:fea0:c066": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
15392023-12-15T01:17:34.839ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::605a:15ff:fea0:c066": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
15402023-12-15T01:17:39.840ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::605a:15ff:fea0:c066": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
15412023-12-15T01:17:44.842ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::605a:15ff:fea0:c066": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
15422023-12-15T01:17:45.056ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:769 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:42399
15432023-12-15T01:17:45.056ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 28 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:42399 req_id = d5d44872-0b82-48df-90a5-305e3e77013b response_code = 200 uri = /baseboard
15442023-12-15T01:17:49.843ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::605a:15ff:fea0:c066": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
15452023-12-15T01:17:54.845ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::605a:15ff:fea0:c066": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
15462023-12-15T01:17:57.992ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 783 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::d]:41278 req_id = 9f2c1d07-9678-4ea6-9fea-1eefd08a11ca response_code = 200 uri = /metrics/collect/78829061-d92a-47ac-bcd6-186a0b9f0582
15472023-12-15T01:17:59.847ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::605a:15ff:fea0:c066": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
15482023-12-15T01:18:04.848ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::605a:15ff:fea0:c066": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
15492023-12-15T01:18:09.851ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::605a:15ff:fea0:c066": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
15502023-12-15T01:18:14.851ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::605a:15ff:fea0:c066": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
15512023-12-15T01:18:15.090ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:769 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:49051
15522023-12-15T01:18:15.090ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 27 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:49051 req_id = c27c49e6-c3b4-4a89-b864-4511ac67b9b6 response_code = 200 uri = /baseboard
15532023-12-15T01:18:19.853ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::605a:15ff:fea0:c066": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
15542023-12-15T01:18:24.854ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::605a:15ff:fea0:c066": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
15552023-12-15T01:18:27.992ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 187 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::d]:41278 req_id = 489ae9fd-a3cf-4f50-81e1-35920c7b97f4 response_code = 200 uri = /metrics/collect/78829061-d92a-47ac-bcd6-186a0b9f0582
15562023-12-15T01:18:29.855ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::605a:15ff:fea0:c066": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
15572023-12-15T01:18:34.857ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::605a:15ff:fea0:c066": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
15582023-12-15T01:18:39.859ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::605a:15ff:fea0:c066": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
15592023-12-15T01:18:44.861ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::605a:15ff:fea0:c066": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
15602023-12-15T01:18:45.125ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:769 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:58799
15612023-12-15T01:18:45.126ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 786 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:58799 req_id = 5fd92e3c-632e-40e8-9209-ffa1c17d0cd6 response_code = 200 uri = /baseboard
15622023-12-15T01:18:49.862ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::605a:15ff:fea0:c066": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
15632023-12-15T01:18:54.863ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::605a:15ff:fea0:c066": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
15642023-12-15T01:18:57.649ZINFOSledAgent (auto-cleanup-task): running automatic periodic zone bundle cleanup file = sled-agent/src/zone_bundle.rs:299
15652023-12-15T01:18:57.649ZINFOSledAgent (StorageManager): Received GetLatestResources(Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) }) file = sled-storage/src/manager.rs:346
15662023-12-15T01:18:57.686ZINFOSledAgent (auto-cleanup-task): top of bundle cleanup loop file = sled-agent/src/zone_bundle.rs:288 next_cleanup = Instant { tv_sec: 2210, tv_nsec: 165104216 } time_to_next_cleanup = 599.99999994s
15672023-12-15T01:18:57.992ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 663 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::d]:41278 req_id = 0af40741-0782-469a-9969-b6a6c1b374e9 response_code = 200 uri = /metrics/collect/78829061-d92a-47ac-bcd6-186a0b9f0582
15682023-12-15T01:18:59.865ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::605a:15ff:fea0:c066": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
15692023-12-15T01:19:04.865ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::605a:15ff:fea0:c066": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
15702023-12-15T01:19:09.868ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::605a:15ff:fea0:c066": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
15712023-12-15T01:19:14.869ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::605a:15ff:fea0:c066": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
15722023-12-15T01:19:15.158ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:769 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:48902
15732023-12-15T01:19:15.159ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 584 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:48902 req_id = 801875fb-285b-4e03-8ce7-c9d16d9ad351 response_code = 200 uri = /baseboard
15742023-12-15T01:19:19.870ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::605a:15ff:fea0:c066": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
15752023-12-15T01:19:24.871ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::605a:15ff:fea0:c066": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
15762023-12-15T01:19:27.991ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 225 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::d]:41278 req_id = f4700520-4c95-4d89-890d-2d4c5dfc2ba1 response_code = 200 uri = /metrics/collect/78829061-d92a-47ac-bcd6-186a0b9f0582
15772023-12-15T01:19:29.872ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::605a:15ff:fea0:c066": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
15782023-12-15T01:19:34.873ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::605a:15ff:fea0:c066": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
15792023-12-15T01:19:39.874ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::605a:15ff:fea0:c066": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
15802023-12-15T01:19:44.253ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:532
15812023-12-15T01:19:44.876ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::605a:15ff:fea0:c066": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
15822023-12-15T01:19:45.191ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:769 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:45336
15832023-12-15T01:19:45.191ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 35 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:45336 req_id = 1ebca038-5d82-475f-a586-49034a521fcd response_code = 200 uri = /baseboard
15842023-12-15T01:19:49.878ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::605a:15ff:fea0:c066": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
15852023-12-15T01:19:54.879ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::605a:15ff:fea0:c066": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
15862023-12-15T01:19:57.992ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 192 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::d]:41278 req_id = b7c4f99d-de64-4182-adb8-e790b309e39b response_code = 200 uri = /metrics/collect/78829061-d92a-47ac-bcd6-186a0b9f0582
15872023-12-15T01:19:59.880ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::605a:15ff:fea0:c066": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
15882023-12-15T01:20:04.882ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::605a:15ff:fea0:c066": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
15892023-12-15T01:20:09.884ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::605a:15ff:fea0:c066": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
15902023-12-15T01:20:14.886ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::605a:15ff:fea0:c066": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
15912023-12-15T01:20:15.224ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:769 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:55505
15922023-12-15T01:20:15.225ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 48 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:55505 req_id = 0afaf3c8-9eb9-4017-8d05-0beb3ca976f9 response_code = 200 uri = /baseboard
15932023-12-15T01:20:19.887ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::605a:15ff:fea0:c066": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
15942023-12-15T01:20:24.888ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::605a:15ff:fea0:c066": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
15952023-12-15T01:20:27.992ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 216 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::d]:41278 req_id = 4c14731c-3d11-4d9d-a409-c24f31bd51a1 response_code = 200 uri = /metrics/collect/78829061-d92a-47ac-bcd6-186a0b9f0582
15962023-12-15T01:20:29.497ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:769 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:52483
15972023-12-15T01:20:29.500ZINFOSledAgent (InstanceManager): ensuring instance is registered file = sled-agent/src/instance_manager.rs:227 hardware = InstanceHardware { properties: InstanceProperties { ncpus: InstanceCpuCount(2), memory: ByteCount(1073741824), hostname: "localshark" }, nics: [NetworkInterface { id: 985afe56-fc39-4541-abcc-7bbac13c05ae, kind: Instance { id: 4aa645b6-9ac6-43e1-afd5-dec8793ba69d }, name: Name("net0"), ip: 172.30.0.6, mac: MacAddr(MacAddr6([168, 64, 37, 241, 0, 130])), subnet: V4(Ipv4Net(Ipv4Network { addr: 172.30.0.0, prefix: 22 })), vni: Vni(2662399), primary: true, slot: 1 }], source_nat: SourceNatConfig { ip: 10.151.2.175, first_port: 0, last_port: 16383 }, ephemeral_ip: Some(10.151.2.176), floating_ips: [], firewall_rules: [VpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 985afe56-fc39-4541-abcc-7bbac13c05ae, kind: Instance { id: 4aa645b6-9ac6-43e1-afd5-dec8793ba69d }, name: Name("net0"), ip: 172.30.0.6, mac: MacAddr(MacAddr6([168, 64, 37, 241, 0, 130])), subnet: V4(Ipv4Net(Ipv4Network { addr: 172.30.0.0, prefix: 22 })), vni: Vni(2662399), primary: true, slot: 1 }], filter_hosts: None, filter_ports: None, filter_protocols: Some([Icmp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, VpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 985afe56-fc39-4541-abcc-7bbac13c05ae, kind: Instance { id: 4aa645b6-9ac6-43e1-afd5-dec8793ba69d }, name: Name("net0"), ip: 172.30.0.6, mac: MacAddr(MacAddr6([168, 64, 37, 241, 0, 130])), subnet: V4(Ipv4Net(Ipv4Network { addr: 172.30.0.0, prefix: 22 })), vni: Vni(2662399), primary: true, slot: 1 }], filter_hosts: Some([Vpc(Vni(2662399))]), filter_ports: None, filter_protocols: None, action: Allow, priority: VpcFirewallRulePriority(65534) }, VpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 985afe56-fc39-4541-abcc-7bbac13c05ae, kind: Instance { id: 4aa645b6-9ac6-43e1-afd5-dec8793ba69d }, name: Name("net0"), ip: 172.30.0.6, mac: MacAddr(MacAddr6([168, 64, 37, 241, 0, 130])), subnet: V4(Ipv4Net(Ipv4Network { addr: 172.30.0.0, prefix: 22 })), vni: Vni(2662399), primary: true, slot: 1 }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(22), last: L4Port(22) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }], dhcp_config: DhcpConfig { dns_servers: [1.1.1.1, 9.9.9.9], host_domain: None, search_domains: [] }, disks: [DiskRequest { device: "nvme", name: "disk-a3aa24d96e2", read_only: false, slot: Slot(1), volume_construction_request: Volume { block_size: 512, id: 3a95ce64-def9-44dc-8fa2-bc68dc014f41, read_only_parent: Some(Volume { block_size: 512, id: 0cb963b7-8b58-403d-8a44-b4dfb0de46ff, read_only_parent: None, sub_volumes: [Region { block_size: 512, blocks_per_extent: 131072, extent_count: 32, gen: 2, opts: CrucibleOpts { cert_pem: None, control: None, flush_timeout: None, id: 9497753d-f735-408e-978f-907f85a56513, key: Some("8ZxOtZ/SWpQK4My+TsL6XYyQUD50QD2DPs6DSW+r59Q="), key_pem: None, lossy: false, read_only: true, root_cert_pem: None, target: ["[fd00:1122:3344:101::16]:19001", "[fd00:1122:3344:101::19]:19001", "[fd00:1122:3344:101::13]:19001"] } }] }), sub_volumes: [Region { block_size: 512, blocks_per_extent: 131072, extent_count: 32, gen: 1, opts: CrucibleOpts { cert_pem: None, control: None, flush_timeout: None, id: 3a95ce64-def9-44dc-8fa2-bc68dc014f41, key: Some("WXrfSbGUVNuKwCHNF7my+CYQfvs9qWy2zHQZEM2mg5k="), key_pem: None, lossy: false, read_only: false, root_cert_pem: None, target: ["[fd00:1122:3344:101::14]:19001", "[fd00:1122:3344:101::18]:19001", "[fd00:1122:3344:101::13]:19002"] } }] } }], cloud_init_bytes: Some("6zyQTVNXSU40LjEAAgEBAAIAAioA+AEAKgBAAAAAAAAAAAAAAAApeFY0EmNpZGF0YSAgICAgRkFUMTIgICAOH75bfKwiwHQLVrQOuwcAzRBe6/Ay5M0WzRnr/lRoaXMgaXMgbm90IGEgYm9vdGFibGUgZGlzay4gIFBsZWFzZSBpbnNlcnQgYSBib290YWJsZSBmbG9wcHkgYW5kDQpwcmVzcyBhbnkga2V5IHRvIHRyeSBhZ2FpbiAuLi4gDQoAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAVar4////DwAAAAAAAAAA8P//////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////D/j///8PAAAAAAAAAADw//////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////8PY2lkYXRhICAgICAIAAAAAAAAAAAAAAAAAAAAAAAAAABBbQBlAHQAYQAtAA8A1GQAYQB0AGEAAAD//wAA/////01FVEEtRH4xICAgAACRjgqPV49XAACOCo9XAgC3AAAAQXUAcwBlAHIALQAPALBkAGEAdABhAAAA//8AAP////9VU0VSLUR+MSAgIAAAko4Kj1ePVwAAjgqPVwAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAHsiaW5zdGFuY2UtaWQiOiI0YWE2NDViNi05YWM2LTQzZTEtYWZkNS1kZWM4NzkzYmE2OWQiLCJsb2NhbC1ob3N0bmFtZSI6ImxvY2Fsc2hhcmsiLCJwdWJsaWMta2V5cyI6WyJzc2gtZWQyNTUxOSBBQUFBQzNOemFDMWxaREkxTlRFNUFBQUFJTXFBeGttQ0w1UHp0cmMvWlJPMW1mdTladHhkRmd0MUtjK2lvdnVTL0xaUiJdfQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA") } instance_id = 4aa645b6-9ac6-43e1-afd5-dec8793ba69d instance_runtime = InstanceRuntimeState { propolis_id: Some(7bfbb4c7-9544-4c62-b705-404d31141456), dst_propolis_id: None, migration_id: None, gen: Generation(3), time_updated: 2023-12-15T01:20:29.087700138Z } propolis_addr = [fd00:1122:3344:101::22]:12400 propolis_id = 7bfbb4c7-9544-4c62-b705-404d31141456 vmm_runtime = VmmRuntimeState { state: Starting, gen: Generation(1), time_updated: 2023-12-15T01:20:29.027846Z }
15982023-12-15T01:20:29.500ZINFOSledAgent (InstanceManager): registering new instance file = sled-agent/src/instance_manager.rs:261 instance_id = 4aa645b6-9ac6-43e1-afd5-dec8793ba69d
15992023-12-15T01:20:29.501ZINFOSledAgent (InstanceManager): initializing new Instance file = sled-agent/src/instance.rs:600 instance_id = 4aa645b6-9ac6-43e1-afd5-dec8793ba69d propolis_id = 7bfbb4c7-9544-4c62-b705-404d31141456 state = InstanceInitialState { hardware: InstanceHardware { properties: InstanceProperties { ncpus: InstanceCpuCount(2), memory: ByteCount(1073741824), hostname: "localshark" }, nics: [NetworkInterface { id: 985afe56-fc39-4541-abcc-7bbac13c05ae, kind: Instance { id: 4aa645b6-9ac6-43e1-afd5-dec8793ba69d }, name: Name("net0"), ip: 172.30.0.6, mac: MacAddr(MacAddr6([168, 64, 37, 241, 0, 130])), subnet: V4(Ipv4Net(Ipv4Network { addr: 172.30.0.0, prefix: 22 })), vni: Vni(2662399), primary: true, slot: 1 }], source_nat: SourceNatConfig { ip: 10.151.2.175, first_port: 0, last_port: 16383 }, ephemeral_ip: Some(10.151.2.176), floating_ips: [], firewall_rules: [VpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 985afe56-fc39-4541-abcc-7bbac13c05ae, kind: Instance { id: 4aa645b6-9ac6-43e1-afd5-dec8793ba69d }, name: Name("net0"), ip: 172.30.0.6, mac: MacAddr(MacAddr6([168, 64, 37, 241, 0, 130])), subnet: V4(Ipv4Net(Ipv4Network { addr: 172.30.0.0, prefix: 22 })), vni: Vni(2662399), primary: true, slot: 1 }], filter_hosts: None, filter_ports: None, filter_protocols: Some([Icmp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, VpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 985afe56-fc39-4541-abcc-7bbac13c05ae, kind: Instance { id: 4aa645b6-9ac6-43e1-afd5-dec8793ba69d }, name: Name("net0"), ip: 172.30.0.6, mac: MacAddr(MacAddr6([168, 64, 37, 241, 0, 130])), subnet: V4(Ipv4Net(Ipv4Network { addr: 172.30.0.0, prefix: 22 })), vni: Vni(2662399), primary: true, slot: 1 }], filter_hosts: Some([Vpc(Vni(2662399))]), filter_ports: None, filter_protocols: None, action: Allow, priority: VpcFirewallRulePriority(65534) }, VpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 985afe56-fc39-4541-abcc-7bbac13c05ae, kind: Instance { id: 4aa645b6-9ac6-43e1-afd5-dec8793ba69d }, name: Name("net0"), ip: 172.30.0.6, mac: MacAddr(MacAddr6([168, 64, 37, 241, 0, 130])), subnet: V4(Ipv4Net(Ipv4Network { addr: 172.30.0.0, prefix: 22 })), vni: Vni(2662399), primary: true, slot: 1 }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(22), last: L4Port(22) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }], dhcp_config: DhcpConfig { dns_servers: [1.1.1.1, 9.9.9.9], host_domain: None, search_domains: [] }, disks: [DiskRequest { device: "nvme", name: "disk-a3aa24d96e2", read_only: false, slot: Slot(1), volume_construction_request: Volume { block_size: 512, id: 3a95ce64-def9-44dc-8fa2-bc68dc014f41, read_only_parent: Some(Volume { block_size: 512, id: 0cb963b7-8b58-403d-8a44-b4dfb0de46ff, read_only_parent: None, sub_volumes: [Region { block_size: 512, blocks_per_extent: 131072, extent_count: 32, gen: 2, opts: CrucibleOpts { cert_pem: None, control: None, flush_timeout: None, id: 9497753d-f735-408e-978f-907f85a56513, key: Some("8ZxOtZ/SWpQK4My+TsL6XYyQUD50QD2DPs6DSW+r59Q="), key_pem: None, lossy: false, read_only: true, root_cert_pem: None, target: ["[fd00:1122:3344:101::16]:19001", "[fd00:1122:3344:101::19]:19001", "[fd00:1122:3344:101::13]:19001"] } }] }), sub_volumes: [Region { block_size: 512, blocks_per_extent: 131072, extent_count: 32, gen: 1, opts: CrucibleOpts { cert_pem: None, control: None, flush_timeout: None, id: 3a95ce64-def9-44dc-8fa2-bc68dc014f41, key: Some("WXrfSbGUVNuKwCHNF7my+CYQfvs9qWy2zHQZEM2mg5k="), key_pem: None, lossy: false, read_only: false, root_cert_pem: None, target: ["[fd00:1122:3344:101::14]:19001", "[fd00:1122:3344:101::18]:19001", "[fd00:1122:3344:101::13]:19002"] } }] } }], cloud_init_bytes: Some("6zyQTVNXSU40LjEAAgEBAAIAAioA+AEAKgBAAAAAAAAAAAAAAAApeFY0EmNpZGF0YSAgICAgRkFUMTIgICAOH75bfKwiwHQLVrQOuwcAzRBe6/Ay5M0WzRnr/lRoaXMgaXMgbm90IGEgYm9vdGFibGUgZGlzay4gIFBsZWFzZSBpbnNlcnQgYSBib290YWJsZSBmbG9wcHkgYW5kDQpwcmVzcyBhbnkga2V5IHRvIHRyeSBhZ2FpbiAuLi4gDQoAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAVar4////DwAAAAAAAAAA8P//////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////D/j///8PAAAAAAAAAADw//////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////8PY2lkYXRhICAgICAIAAAAAAAAAAAAAAAAAAAAAAAAAABBbQBlAHQAYQAtAA8A1GQAYQB0AGEAAAD//wAA/////01FVEEtRH4xICAgAACRjgqPV49XAACOCo9XAgC3AAAAQXUAcwBlAHIALQAPALBkAGEAdABhAAAA//8AAP////9VU0VSLUR+MSAgIAAAko4Kj1ePVwAAjgqPVwAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAHsiaW5zdGFuY2UtaWQiOiI0YWE2NDViNi05YWM2LTQzZTEtYWZkNS1kZWM4NzkzYmE2OWQiLCJsb2NhbC1ob3N0bmFtZSI6ImxvY2Fsc2hhcmsiLCJwdWJsaWMta2V5cyI6WyJzc2gtZWQyNTUxOSBBQUFBQzNOemFDMWxaREkxTlRFNUFBQUFJTXFBeGttQ0w1UHp0cmMvWlJPMW1mdTladHhkRmd0MUtjK2lvdnVTL0xaUiJdfQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA") }, instance_runtime: InstanceRuntimeState { propolis_id: Some(7bfbb4c7-9544-4c62-b705-404d31141456), dst_propolis_id: None, migration_id: None, gen: Generation(3), time_updated: 2023-12-15T01:20:29.087700138Z }, vmm_runtime: VmmRuntimeState { state: Starting, gen: Generation(1), time_updated: 2023-12-15T01:20:29.027846Z }, propolis_addr: [fd00:1122:3344:101::22]:12400 }
16002023-12-15T01:20:29.501ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 3735 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:52483 req_id = 9406c23c-9ffd-4458-858d-73a7f27891b7 response_code = 200 uri = /instances/4aa645b6-9ac6-43e1-afd5-dec8793ba69d
16012023-12-15T01:20:29.628ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:769 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:46690
16022023-12-15T01:20:29.649ZINFOSledAgent (PortManager): Created OPTE port file = illumos-utils/src/opte/port_manager.rs:396 port = Port { inner: PortInner { name: "opte6", _ip: 172.30.0.6, mac: MacAddr6([168, 64, 37, 241, 0, 130]), slot: 1, vni: Vni { inner: 2662399 }, gateway: Gateway { mac: MacAddr6([168, 64, 37, 255, 119, 119]), ip: 172.30.0.1 }, vnic: "vopte6" } }
16032023-12-15T01:20:29.649ZINFOSledAgent (StorageManager): Received GetLatestResources(Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) }) file = sled-storage/src/manager.rs:346
16042023-12-15T01:20:29.684ZINFOSledAgent (InstanceManager): Configuring new Omicron zone: oxz_propolis-server_7bfbb4c7-9544-4c62-b705-404d31141456 file = illumos-utils/src/zone.rs:314 instance_id = 4aa645b6-9ac6-43e1-afd5-dec8793ba69d
16052023-12-15T01:20:29.704ZINFOSledAgent (InstanceManager): Installing Omicron zone: oxz_propolis-server_7bfbb4c7-9544-4c62-b705-404d31141456 file = illumos-utils/src/zone.rs:353 instance_id = 4aa645b6-9ac6-43e1-afd5-dec8793ba69d
16062023-12-15T01:20:29.890ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::605a:15ff:fea0:c066": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
16072023-12-15T01:20:31.228ZINFOSledAgent (InstanceManager): Profile for oxz_propolis-server_7bfbb4c7-9544-4c62-b705-404d31141456: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="system/illumos/propolis-server"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value="oxControlInstance0"/> <propval type="astring" name="gateway" value="fd00:1122:3344:101::1"/> <propval type="astring" name="listen_addr" value="fd00:1122:3344:101::22"/> <propval type="astring" name="listen_port" value="12400"/> <propval type="astring" name="metric_addr" value="[fd00:1122:3344:101::a]:12221"/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34 instance_id = 4aa645b6-9ac6-43e1-afd5-dec8793ba69d
16082023-12-15T01:20:31.228ZINFOSledAgent (InstanceManager): Zone booting file = illumos-utils/src/running_zone.rs:512 instance_id = 4aa645b6-9ac6-43e1-afd5-dec8793ba69d zone = oxz_propolis-server_7bfbb4c7-9544-4c62-b705-404d31141456
16092023-12-15T01:20:31.411ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_7bfbb4c7-9544-4c62-b705-404d31141456") failed: Property not found. retry in 28.289973ms file = illumos-utils/src/svc.rs:36 instance_id = 4aa645b6-9ac6-43e1-afd5-dec8793ba69d zone = oxz_propolis-server_7bfbb4c7-9544-4c62-b705-404d31141456
16102023-12-15T01:20:31.449ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_7bfbb4c7-9544-4c62-b705-404d31141456") failed: Property not found. retry in 62.186374ms file = illumos-utils/src/svc.rs:36 instance_id = 4aa645b6-9ac6-43e1-afd5-dec8793ba69d zone = oxz_propolis-server_7bfbb4c7-9544-4c62-b705-404d31141456
16112023-12-15T01:20:31.528ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_7bfbb4c7-9544-4c62-b705-404d31141456") failed: Property not found. retry in 103.820309ms file = illumos-utils/src/svc.rs:36 instance_id = 4aa645b6-9ac6-43e1-afd5-dec8793ba69d zone = oxz_propolis-server_7bfbb4c7-9544-4c62-b705-404d31141456
16122023-12-15T01:20:31.651ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_7bfbb4c7-9544-4c62-b705-404d31141456") failed: Property not found. retry in 205.892799ms file = illumos-utils/src/svc.rs:36 instance_id = 4aa645b6-9ac6-43e1-afd5-dec8793ba69d zone = oxz_propolis-server_7bfbb4c7-9544-4c62-b705-404d31141456
16132023-12-15T01:20:31.867ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_7bfbb4c7-9544-4c62-b705-404d31141456") failed: Property not found. retry in 978.34608ms file = illumos-utils/src/svc.rs:36 instance_id = 4aa645b6-9ac6-43e1-afd5-dec8793ba69d zone = oxz_propolis-server_7bfbb4c7-9544-4c62-b705-404d31141456
16142023-12-15T01:20:32.858ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_7bfbb4c7-9544-4c62-b705-404d31141456") failed: Property not found. retry in 923.686077ms file = illumos-utils/src/svc.rs:36 instance_id = 4aa645b6-9ac6-43e1-afd5-dec8793ba69d zone = oxz_propolis-server_7bfbb4c7-9544-4c62-b705-404d31141456
16152023-12-15T01:20:33.793ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_7bfbb4c7-9544-4c62-b705-404d31141456") failed: Property not found. retry in 884.033098ms file = illumos-utils/src/svc.rs:36 instance_id = 4aa645b6-9ac6-43e1-afd5-dec8793ba69d zone = oxz_propolis-server_7bfbb4c7-9544-4c62-b705-404d31141456
16162023-12-15T01:20:34.688ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_7bfbb4c7-9544-4c62-b705-404d31141456") failed: Property not found. retry in 1.193799997s file = illumos-utils/src/svc.rs:36 instance_id = 4aa645b6-9ac6-43e1-afd5-dec8793ba69d zone = oxz_propolis-server_7bfbb4c7-9544-4c62-b705-404d31141456
16172023-12-15T01:20:34.892ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::605a:15ff:fea0:c066": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
16182023-12-15T01:20:35.912ZINFOSledAgent (InstanceManager): Started propolis in zone: oxz_propolis-server_7bfbb4c7-9544-4c62-b705-404d31141456 file = sled-agent/src/instance.rs:991 instance_id = 4aa645b6-9ac6-43e1-afd5-dec8793ba69d
16192023-12-15T01:20:35.921ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_7bfbb4c7-9544-4c62-b705-404d31141456") failed: Property not found. retry in 35.89433ms file = illumos-utils/src/svc.rs:36 instance_id = 4aa645b6-9ac6-43e1-afd5-dec8793ba69d
16202023-12-15T01:20:35.967ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_7bfbb4c7-9544-4c62-b705-404d31141456") failed: Property not found. retry in 123.842523ms file = illumos-utils/src/svc.rs:36 instance_id = 4aa645b6-9ac6-43e1-afd5-dec8793ba69d
16212023-12-15T01:20:36.102ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_7bfbb4c7-9544-4c62-b705-404d31141456") failed: Property not found. retry in 236.07233ms file = illumos-utils/src/svc.rs:36 instance_id = 4aa645b6-9ac6-43e1-afd5-dec8793ba69d
16222023-12-15T01:20:36.348ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_7bfbb4c7-9544-4c62-b705-404d31141456") failed: Property not found. retry in 447.023887ms file = illumos-utils/src/svc.rs:36 instance_id = 4aa645b6-9ac6-43e1-afd5-dec8793ba69d
16232023-12-15T01:20:36.806ZINFOSledAgent (InstanceManager): Propolis SMF service is online file = sled-agent/src/instance.rs:1000 instance_id = 4aa645b6-9ac6-43e1-afd5-dec8793ba69d
16242023-12-15T01:20:36.841ZINFOSledAgent (InstanceManager): Propolis HTTP server online file = sled-agent/src/instance.rs:1014 instance_id = 4aa645b6-9ac6-43e1-afd5-dec8793ba69d
16252023-12-15T01:20:36.841ZINFOSledAgent (InstanceManager): Sending ensure request to propolis: InstanceEnsureRequest { cloud_init_bytes: Some("6zyQTVNXSU40LjEAAgEBAAIAAioA+AEAKgBAAAAAAAAAAAAAAAApeFY0EmNpZGF0YSAgICAgRkFUMTIgICAOH75bfKwiwHQLVrQOuwcAzRBe6/Ay5M0WzRnr/lRoaXMgaXMgbm90IGEgYm9vdGFibGUgZGlzay4gIFBsZWFzZSBpbnNlcnQgYSBib290YWJsZSBmbG9wcHkgYW5kDQpwcmVzcyBhbnkga2V5IHRvIHRyeSBhZ2FpbiAuLi4gDQoAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAVar4////DwAAAAAAAAAA8P//////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////D/j///8PAAAAAAAAAADw//////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////8PY2lkYXRhICAgICAIAAAAAAAAAAAAAAAAAAAAAAAAAABBbQBlAHQAYQAtAA8A1GQAYQB0AGEAAAD//wAA/////01FVEEtRH4xICAgAACRjgqPV49XAACOCo9XAgC3AAAAQXUAcwBlAHIALQAPALBkAGEAdABhAAAA//8AAP////9VU0VSLUR+MSAgIAAAko4Kj1ePVwAAjgqPVwAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAHsiaW5zdGFuY2UtaWQiOiI0YWE2NDViNi05YWM2LTQzZTEtYWZkNS1kZWM4NzkzYmE2OWQiLCJsb2NhbC1ob3N0bmFtZSI6ImxvY2Fsc2hhcmsiLCJwdWJsaWMta2V5cyI6WyJzc2gtZWQyNTUxOSBBQUFBQzNOemFDMWxaREkxTlRFNUFBQUFJTXFBeGttQ0w1UHp0cmMvWlJPMW1mdTladHhkRmd0MUtjK2lvdnVTL0xaUiJdfQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA"), disks: [DiskRequest { device: "nvme", name: "disk-a3aa24d96e2", read_only: false, slot: Slot(1), volume_construction_request: Volume { block_size: 512, id: 3a95ce64-def9-44dc-8fa2-bc68dc014f41, read_only_parent: Some(Volume { block_size: 512, id: 0cb963b7-8b58-403d-8a44-b4dfb0de46ff, read_only_parent: None, sub_volumes: [Region { block_size: 512, blocks_per_extent: 131072, extent_count: 32, gen: 2, opts: CrucibleOpts { cert_pem: None, control: None, flush_timeout: None, id: 9497753d-f735-408e-978f-907f85a56513, key: Some("8ZxOtZ/SWpQK4My+TsL6XYyQUD50QD2DPs6DSW+r59Q="), key_pem: None, lossy: false, read_only: true, root_cert_pem: None, target: ["[fd00:1122:3344:101::16]:19001", "[fd00:1122:3344:101::19]:19001", "[fd00:1122:3344:101::13]:19001"] } }] }), sub_volumes: [Region { block_size: 512, blocks_per_extent: 131072, extent_count: 32, gen: 1, opts: CrucibleOpts { cert_pem: None, control: None, flush_timeout: None, id: 3a95ce64-def9-44dc-8fa2-bc68dc014f41, key: Some("WXrfSbGUVNuKwCHNF7my+CYQfvs9qWy2zHQZEM2mg5k="), key_pem: None, lossy: false, read_only: false, root_cert_pem: None, target: ["[fd00:1122:3344:101::14]:19001", "[fd00:1122:3344:101::18]:19001", "[fd00:1122:3344:101::13]:19002"] } }] } }], migrate: None, nics: [NetworkInterfaceRequest { name: "vopte6", slot: Slot(1) }], properties: InstanceProperties { bootrom_id: 00000000-0000-0000-0000-000000000000, description: "Test description", id: 4aa645b6-9ac6-43e1-afd5-dec8793ba69d, image_id: 00000000-0000-0000-0000-000000000000, memory: 1024, name: "localshark", vcpus: 2 } } file = sled-agent/src/instance.rs:451 instance_id = 4aa645b6-9ac6-43e1-afd5-dec8793ba69d
16262023-12-15T01:20:37.020ZINFOSledAgent (InstanceManager): result of instance_ensure call is Ok(InstanceEnsureResponse { migrate: None }) file = sled-agent/src/instance.rs:453 instance_id = 4aa645b6-9ac6-43e1-afd5-dec8793ba69d
16272023-12-15T01:20:37.021ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 7392802 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:46690 req_id = 979fb57b-d580-4570-8254-ccfcc9180436 response_code = 200 uri = /instances/4aa645b6-9ac6-43e1-afd5-dec8793ba69d/state
16282023-12-15T01:20:37.021ZINFOSledAgent (InstanceManager): Observing new propolis state: ObservedPropolisState { vmm_state: PropolisInstanceState(Starting), migration_status: NoMigration, time: 2023-12-15T01:20:37.021155857Z } file = sled-agent/src/instance.rs:335 instance_id = 4aa645b6-9ac6-43e1-afd5-dec8793ba69d
16292023-12-15T01:20:37.021ZINFOSledAgent (InstanceManager): updated state after observing Propolis state change file = sled-agent/src/instance.rs:357 instance_id = 4aa645b6-9ac6-43e1-afd5-dec8793ba69d new_instance_state = InstanceRuntimeState { propolis_id: Some(7bfbb4c7-9544-4c62-b705-404d31141456), dst_propolis_id: None, migration_id: None, gen: Generation(3), time_updated: 2023-12-15T01:20:29.087700138Z } new_vmm_state = VmmRuntimeState { state: Starting, gen: Generation(2), time_updated: 2023-12-15T01:20:37.021155857Z } propolis_id = 7bfbb4c7-9544-4c62-b705-404d31141456
16302023-12-15T01:20:37.021ZINFOSledAgent (InstanceManager): Publishing instance state update to Nexus file = sled-agent/src/instance.rs:260 instance_id = 4aa645b6-9ac6-43e1-afd5-dec8793ba69d state = SledInstanceState { instance_state: InstanceRuntimeState { propolis_id: Some(7bfbb4c7-9544-4c62-b705-404d31141456), dst_propolis_id: None, migration_id: None, gen: Generation(3), time_updated: 2023-12-15T01:20:29.087700138Z }, propolis_id: 7bfbb4c7-9544-4c62-b705-404d31141456, vmm_state: VmmRuntimeState { state: Starting, gen: Generation(2), time_updated: 2023-12-15T01:20:37.021155857Z } }
16312023-12-15T01:20:37.100ZINFOSledAgent (InstanceManager): Observing new propolis state: ObservedPropolisState { vmm_state: PropolisInstanceState(Running), migration_status: NoMigration, time: 2023-12-15T01:20:37.100973639Z } file = sled-agent/src/instance.rs:335 instance_id = 4aa645b6-9ac6-43e1-afd5-dec8793ba69d
16322023-12-15T01:20:37.101ZINFOSledAgent (InstanceManager): updated state after observing Propolis state change file = sled-agent/src/instance.rs:357 instance_id = 4aa645b6-9ac6-43e1-afd5-dec8793ba69d new_instance_state = InstanceRuntimeState { propolis_id: Some(7bfbb4c7-9544-4c62-b705-404d31141456), dst_propolis_id: None, migration_id: None, gen: Generation(3), time_updated: 2023-12-15T01:20:29.087700138Z } new_vmm_state = VmmRuntimeState { state: Running, gen: Generation(3), time_updated: 2023-12-15T01:20:37.100973639Z } propolis_id = 7bfbb4c7-9544-4c62-b705-404d31141456
16332023-12-15T01:20:37.101ZINFOSledAgent (InstanceManager): Publishing instance state update to Nexus file = sled-agent/src/instance.rs:260 instance_id = 4aa645b6-9ac6-43e1-afd5-dec8793ba69d state = SledInstanceState { instance_state: InstanceRuntimeState { propolis_id: Some(7bfbb4c7-9544-4c62-b705-404d31141456), dst_propolis_id: None, migration_id: None, gen: Generation(3), time_updated: 2023-12-15T01:20:29.087700138Z }, propolis_id: 7bfbb4c7-9544-4c62-b705-404d31141456, vmm_state: VmmRuntimeState { state: Running, gen: Generation(3), time_updated: 2023-12-15T01:20:37.100973639Z } }
16342023-12-15T01:20:39.893ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::605a:15ff:fea0:c066": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
16352023-12-15T01:20:44.893ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::605a:15ff:fea0:c066": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
16362023-12-15T01:20:45.258ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:769 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:39446
16372023-12-15T01:20:45.259ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 544 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:39446 req_id = 24bb20b0-8c11-46e2-8274-318d1e0c4746 response_code = 200 uri = /baseboard
16382023-12-15T01:20:49.896ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::605a:15ff:fea0:c066": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
16392023-12-15T01:20:54.158ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:769 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:63204
16402023-12-15T01:20:54.158ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 470 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:63204 req_id = 99944bef-109d-421c-94e6-861444e72586 response_code = 200 uri = /instances/4aa645b6-9ac6-43e1-afd5-dec8793ba69d/state
16412023-12-15T01:20:54.158ZINFOSledAgent (InstanceManager): Observing new propolis state: ObservedPropolisState { vmm_state: PropolisInstanceState(Stopping), migration_status: NoMigration, time: 2023-12-15T01:20:54.158755402Z } file = sled-agent/src/instance.rs:335 instance_id = 4aa645b6-9ac6-43e1-afd5-dec8793ba69d
16422023-12-15T01:20:54.158ZINFOSledAgent (InstanceManager): updated state after observing Propolis state change file = sled-agent/src/instance.rs:357 instance_id = 4aa645b6-9ac6-43e1-afd5-dec8793ba69d new_instance_state = InstanceRuntimeState { propolis_id: Some(7bfbb4c7-9544-4c62-b705-404d31141456), dst_propolis_id: None, migration_id: None, gen: Generation(3), time_updated: 2023-12-15T01:20:29.087700138Z } new_vmm_state = VmmRuntimeState { state: Stopping, gen: Generation(5), time_updated: 2023-12-15T01:20:54.158755402Z } propolis_id = 7bfbb4c7-9544-4c62-b705-404d31141456
16432023-12-15T01:20:54.158ZINFOSledAgent (InstanceManager): Publishing instance state update to Nexus file = sled-agent/src/instance.rs:260 instance_id = 4aa645b6-9ac6-43e1-afd5-dec8793ba69d state = SledInstanceState { instance_state: InstanceRuntimeState { propolis_id: Some(7bfbb4c7-9544-4c62-b705-404d31141456), dst_propolis_id: None, migration_id: None, gen: Generation(3), time_updated: 2023-12-15T01:20:29.087700138Z }, propolis_id: 7bfbb4c7-9544-4c62-b705-404d31141456, vmm_state: VmmRuntimeState { state: Stopping, gen: Generation(5), time_updated: 2023-12-15T01:20:54.158755402Z } }
16442023-12-15T01:20:54.213ZINFOSledAgent (InstanceManager): Observing new propolis state: ObservedPropolisState { vmm_state: PropolisInstanceState(Stopped), migration_status: NoMigration, time: 2023-12-15T01:20:54.213537297Z } file = sled-agent/src/instance.rs:335 instance_id = 4aa645b6-9ac6-43e1-afd5-dec8793ba69d
16452023-12-15T01:20:54.213ZINFOSledAgent (InstanceManager): updated state after observing Propolis state change file = sled-agent/src/instance.rs:357 instance_id = 4aa645b6-9ac6-43e1-afd5-dec8793ba69d new_instance_state = InstanceRuntimeState { propolis_id: Some(7bfbb4c7-9544-4c62-b705-404d31141456), dst_propolis_id: None, migration_id: None, gen: Generation(3), time_updated: 2023-12-15T01:20:29.087700138Z } new_vmm_state = VmmRuntimeState { state: Stopping, gen: Generation(6), time_updated: 2023-12-15T01:20:54.213537297Z } propolis_id = 7bfbb4c7-9544-4c62-b705-404d31141456
16462023-12-15T01:20:54.213ZINFOSledAgent (InstanceManager): Publishing instance state update to Nexus file = sled-agent/src/instance.rs:260 instance_id = 4aa645b6-9ac6-43e1-afd5-dec8793ba69d state = SledInstanceState { instance_state: InstanceRuntimeState { propolis_id: Some(7bfbb4c7-9544-4c62-b705-404d31141456), dst_propolis_id: None, migration_id: None, gen: Generation(3), time_updated: 2023-12-15T01:20:29.087700138Z }, propolis_id: 7bfbb4c7-9544-4c62-b705-404d31141456, vmm_state: VmmRuntimeState { state: Stopping, gen: Generation(6), time_updated: 2023-12-15T01:20:54.213537297Z } }
16472023-12-15T01:20:54.351ZINFOSledAgent (InstanceManager): Observing new propolis state: ObservedPropolisState { vmm_state: PropolisInstanceState(Destroyed), migration_status: NoMigration, time: 2023-12-15T01:20:54.351732428Z } file = sled-agent/src/instance.rs:335 instance_id = 4aa645b6-9ac6-43e1-afd5-dec8793ba69d
16482023-12-15T01:20:54.351ZINFOSledAgent (InstanceManager): updated state after observing Propolis state change file = sled-agent/src/instance.rs:357 instance_id = 4aa645b6-9ac6-43e1-afd5-dec8793ba69d new_instance_state = InstanceRuntimeState { propolis_id: None, dst_propolis_id: None, migration_id: None, gen: Generation(5), time_updated: 2023-12-15T01:20:54.351732428Z } new_vmm_state = VmmRuntimeState { state: Destroyed, gen: Generation(7), time_updated: 2023-12-15T01:20:54.351732428Z } propolis_id = 7bfbb4c7-9544-4c62-b705-404d31141456
16492023-12-15T01:20:54.351ZINFOSledAgent (InstanceManager): terminating VMM that has exited file = sled-agent/src/instance.rs:372 instance_id = 4aa645b6-9ac6-43e1-afd5-dec8793ba69d
16502023-12-15T01:20:54.351ZINFOSledAgent (StorageManager): Received GetLatestResources(Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) }) file = sled-storage/src/manager.rs:346
16512023-12-15T01:20:54.351ZINFOSledAgent (StorageManager): Received GetLatestResources(Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: true, is_tx_task_set: false } }) }) file = sled-storage/src/manager.rs:346
16522023-12-15T01:20:54.352ZINFOSledAgent (ZoneBundler): creating zone bundle context = ZoneBundleContext { storage_dirs: ["/pool/int/a462a7f7-b628-40fe-80ff-4e4189e2d62b/debug/bundle/zone", "/pool/int/b462a7f7-b628-40fe-80ff-4e4189e2d62b/debug/bundle/zone"], cause: TerminatedInstance, extra_log_dirs: ["/pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/debug", "/pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/debug", "/pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/debug", "/pool/ext/616b26df-e62a-4c68-b506-f4a923d8aaf7/crypt/debug", "/pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/debug", "/pool/ext/ceb4461c-cf56-4719-ad3c-14430bfdfb60/crypt/debug", "/pool/ext/d462a7f7-b628-40fe-80ff-4e4189e2d62b/crypt/debug", "/pool/ext/e4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/debug", "/pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/debug"] } file = sled-agent/src/zone_bundle.rs:436 zone_name = oxz_propolis-server_7bfbb4c7-9544-4c62-b705-404d31141456
16532023-12-15T01:20:54.897ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::605a:15ff:fea0:c066": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}
16542023-12-15T01:20:54.964ZINFOSledAgent (ZoneBundler): finished zone bundle file = sled-agent/src/zone_bundle.rs:1229 metadata = ZoneBundleMetadata { id: ZoneBundleId { zone_name: "oxz_propolis-server_7bfbb4c7-9544-4c62-b705-404d31141456", bundle_id: 4f7fa57a-de80-4855-ae5d-60374a564f99 }, time_created: 2023-12-15T01:20:54.352216996Z, version: 0, cause: TerminatedInstance }
16552023-12-15T01:20:54.964ZWARNSledAgent (InstanceManager): Halting and removing zone: oxz_propolis-server_7bfbb4c7-9544-4c62-b705-404d31141456 file = sled-agent/src/instance.rs:545 instance_id = 4aa645b6-9ac6-43e1-afd5-dec8793ba69d
16562023-12-15T01:20:56.536ZINFOSledAgent (InstanceManager): halt_and_remove_logged: Previous zone state: Running file = illumos-utils/src/zone.rs:268 instance_id = 4aa645b6-9ac6-43e1-afd5-dec8793ba69d
16572023-12-15T01:20:56.562ZINFOSledAgent (InstanceManager): Publishing instance state update to Nexus file = sled-agent/src/instance.rs:260 instance_id = 4aa645b6-9ac6-43e1-afd5-dec8793ba69d state = SledInstanceState { instance_state: InstanceRuntimeState { propolis_id: None, dst_propolis_id: None, migration_id: None, gen: Generation(5), time_updated: 2023-12-15T01:20:54.351732428Z }, propolis_id: 7bfbb4c7-9544-4c62-b705-404d31141456, vmm_state: VmmRuntimeState { state: Destroyed, gen: Generation(7), time_updated: 2023-12-15T01:20:54.351732428Z } }
16582023-12-15T01:20:56.636ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:769 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:52768
16592023-12-15T01:20:56.636ZWARNSledAgent (PortManager): unset_virtual_nic_host unimplmented file = illumos-utils/src/opte/port_manager.rs:507
16602023-12-15T01:20:56.636ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 452 local_addr = [fd00:1122:3344:101::1]:12345 method = DELETE remote_addr = [fd00:1122:3344:101::c]:52768 req_id = ea64650c-04b0-4a01-b68a-1be990224ca2 response_code = 204 uri = /v2p/985afe56-fc39-4541-abcc-7bbac13c05ae
16612023-12-15T01:20:56.857ZINFOSledAgent (InstanceManager): State monitoring task complete file = sled-agent/src/instance.rs:488 instance_id = 4aa645b6-9ac6-43e1-afd5-dec8793ba69d
16622023-12-15T01:20:57.992ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 381 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::d]:41278 req_id = fa2f089e-7ca1-40ca-abf5-a1761d2f23e6 response_code = 200 uri = /metrics/collect/78829061-d92a-47ac-bcd6-186a0b9f0582
16632023-12-15T01:20:59.899ZINFOSledAgent (bootstore_ddmd_poller): Received prefixes from ddmd DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:119 prefixes = {"fe80::605a:15ff:fea0:c066": [PathVector { destination: Ipv6Prefix { addr: fd00:99::, len: 64 }, path: ["oxz_switch"] }]}